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I.  INTRODUCTION 


A.  BACKGROUND 

During  a  time  period  when  the  discussion  of  defense  budget  cuts  and 
sequestration  dominates  the  pages  of  national  and  defense  news,  what  strategy  is  the 
military  employing  to  operate  more  efficiently?  The  wars  in  Iraq  and  Afghanistan  are 
coming  to  a  close,  and  with  that,  a  further  reduction  in  operating  budgets  is  expected 
across  the  Department  of  Defense  (DoD).  The  need  for  a  prudent  and  accountable 
expenditure  of  taxpayer  dollars  has  reached  its  pinnacle  with  the  requirement  for  the  DoD 
to  be  100%  auditable  by  2017,  per  Secretary  of  Defense  Panetta’s  direction.  The  priority 
of  fiscal  responsibility  has  gone  so  far  as  to  be  embedded  in  the  National  Security 
Strategy,  as  stated  in  the  following  excerpt: 

Cost-effective  and  efficient  processes  are  particularly  important  for  the 
Department  of  Defense,  which  accounts  for  approximately  70  percent  of 
all  Federal  procurement  spending.  We  will  scrutinize  our  programs  and 
tenninate  or  restructure  those  that  are  outdated,  duplicative,  ineffective,  or 
wasteful.  The  result  will  be  more  relevant,  capable,  and  effective 
programs  and  systems  that  our  military  wants  and  needs.  (President  of  the 
United  States,  2010,  p.  34) 

In  addition  to  spending  funds  more  wisely,  further  motivation  behind  this  research 
is  to  improve  the  overall  condition  of  readiness  experienced  across  the  Navy’s  fleet. 
Readiness  is  measured  in  various  ways,  ranging  from  detailed  ship-wide  assessments  to 
the  submission  of  an  individual  casualty  report  (CASREP).  A  CASREP  is  a  report  made 
to  the  ship’s  operational  chain  of  command  concerning  a  significant  equipment 
malfunction  that  cannot  be  corrected  within  a  48-hour  period.  Due  to  the  frequency  and 
specific  nature  of  CASREPs,  these  reports  played  a  significant  role  in  the  analysis 
discussed  in  this  paper. 

In  a  2012  report,  the  Government  Accountability  Office  (GAO)  provided 
evidence,  with  results  displayed  as  follows  in  Figure  1,  concerning  a  noticeable  increase 
in  the  number  of  CASREPs  reported  from  2008  to  2012;  as  the  GAO  noted,  an  increase 
in  the  number  of  CASREPs  would  indicate  a  decline  in  overall  material  readiness  (GAO, 
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2012).  In  our  research,  we  used  CASREP  frequency  and  severity  as  a  measure  of 
effectiveness,  and  most  important,  we  considered  only  those  CASREPs  that  require 
additional  repair  parts  due  to  onboard  allowance  shortages. 


Q1  Q2  Q3  Q4  Q1  Q2  Q3  Q4  Q1  Q2  Q3  Q4  Q1  Q2  Q3  Q4  Q1 
2008  2009  2010  |  2011  |  2012 

■  Surface  Combatants  ■  Amphibious  Warfare  Ships 


Figure  1 .  Average  Number  of  Casualty  Reports  of  Surface  Combatant  and 
Amphibious  Warfare  Ships  by  Quarter  From  January  2008  Through  March  2012 

For  the  U.S.  military  to  remain  effective  as  funding  is  reduced,  efficiencies  and 
cost-saving  opportunities  must  be  identified  and  implemented  to  ensure  that  the  nation 
can  sustain  a  competitive  advantage  in  the  21st  century.  There  are  myriad  ways  that  the 
DoD  can  achieve  cost  savings  from  the  numerous  programs  in  development,  as  well  as 
from  those  that  have  reached  maturity  and  are  currently  in  the  sustainment  phase.  In  this 
research,  we  seek  to  identify  one  such  avenue  for  the  U.S.  Navy  that  would  require  a 
minimal  investment  and  result  in  maximum  effectiveness. 

In  addition  to  the  fact  that  the  DoD  has  been  mandated  to  reduce  spending  and 
find  efficiencies  wherever  possible,  other  significant  reasons  justify  why  such  research 
should  take  place.  Following  the  events  of  September  11,  the  entire  DoD  saw  a  dramatic 
increase  in  the  amount  of  funding  appropriated  for  operations  and  maintenance  (O&M; 
OMB,  2003).  This  O&M  funding  is  used  to  pay  for  the  day-to-day  activities  of  our 
deployed  or  deployable  forces  and  typically  consists  of  the  costs  associated  with  fuel, 
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repair  parts,  and  maintenance.  The  increase  in  O&M  funding  resulted  in  a  decrease  in 
acquisition  funding  that  covers  the  research,  development,  and  production  of  the  next 
generation  of  weapons  systems.  As  the  campaigns  in  Afghanistan  and  Iraq  expire,  O&M 
funding  is  expected  to  decrease  and  return  to  more  traditional,  peacetime  levels. 
However,  the  expected  decrease  in  O&M  funding  does  not  guarantee  a  subsequent 
increase  in  procurement  funding.  This  distinction  is  important  to  note,  since  the  findings 
of  the  present  research  may  impact  both  appropriation  types  and,  if  successful,  would 
result  in  a  reduction  of  total  ownership  cost  (TOC). 

According  to  the  Defense  Acquisition  University  (DAU,  2012),  TOC  is  defined  as 
a  concept  designed  to  determine  the  true  cost  of  the  design,  development,  ownership,  and 
support  of  DoD  weapons  systems.  Within  TOC  are  the  acquisition  costs  and  operations 
and  sustaimnent  (O&S)  costs  of  the  system,  with  the  O&S  costs  accounting  for  a 
significantly  larger  segment.  Figure  2  presents  the  breakdown  of  TOC  of  a  weapons 
system  with  greater  clarity. 


Figure  2.  Total  Own 

ership  Cost  Throughout  the  System  Life  Cycle 
(From  Defense  Systems  Management  College,  1997,  p.  181) 
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Figure  2  makes  clear,  and  the  GAO  has  agreed,  that  significantly  more  cost 
savings  can  be  generated  throughout  the  O&S  phase  as  a  result  of  more  aggressive 
research  during  the  procurement  phase  of  the  system  life  cycle  (GAO,  2010).  A 
reduction  in  O&S  costs  translates  directly  to  a  reduction  in  the  amount  of  O&M  funding 
required  for  agencies  to  operate,  which  further  results  in  an  overall  savings  by  the  DoD. 
As  mentioned  previously,  a  reduction  in  O&M  funding  would  provide  the  opportunity  to 
invest  more  in  the  acquisition  of  critical  programs  and  potentially  improve  our  national 
defense  capabilities.  Our  research  focuses  on  the  O&S  phase,  since  this  is  the  phase  that 
requires  the  largest  amount  of  funding. 

B.  SCOPE 

The  scope  of  this  project  is  to  conduct  a  business  case  analysis  (BCA)  of  the 
Continuous  Integrated  Logistics  Support-Targeted  Allowancing  Technique  (CILS— TAT) 
with  the  assistance  of  the  Naval  Sea  Logistics  Center  (NSLC).  According  to  the  NSLC 
command  description,  “NSLC  is  a  field  activity  of  the  Naval  Undersea  Warfare  Center, 
and  is  tasked  with  providing  integrated  logistics,  engineering,  and  information  technology 
expertise  to  all  facets  of  the  Navy's  worldwide  logistics  support  structure”  (NSLC,  2012). 

Embedded  in  NSLC’s  integrated  logistics  support  responsibilities  are  the 
requirements  to  manage  the  configuration  of  naval  weapons  systems  within  the  different 
classes  of  ships  and  submarines  as  well  as  to  provide  the  appropriate  allowances  for  spare 
parts  to  be  stocked  onboard.  Configuration  management,  while  closely  related  to  the 
allowancing  process,  is  beyond  the  scope  of  our  research  and  therefore  is  not  fully 
introduced  here.  Our  research  instead  focuses  on  a  subset  of  the  allowancing  process 
known  as  CILS-TAT.  We  discuss  this  technique,  as  well  as  the  traditional  allowancing 
process,  in  much  greater  detail  in  subsequent  chapters  and  provide  a  better  understanding 
of  the  variables  considered  when  NSLC  is  determining  allowances. 

Our  current  motivation  behind  evaluating  the  effectiveness  of  the  CILS-TAT  is 
the  suspension  of  the  program  since  2009,  when  the  U.S.  Navy  began  its  transition  to  an 
enterprise  resource  planning  (ERP)  framework.  We  expect  that  by  the  time  this  research 
is  published,  the  U.S.  Navy’s  allowancing  process  systems  will  have  fully  transitioned 
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into  the  ERP  framework;  therefore,  the  time  for  a  possible  re-implementation  of  the 
CILS-TAT  is  quickly  approaching.  Based  on  our  findings,  we  hope  to  demonstrate  the 
level  of  effectiveness  achieved  as  a  result  of  using  the  CILS-TAT  and  provide 
recommendations  concerning  the  use  of  the  program  in  the  future. 

The  CILS-TAT  can  be  used  onboard  any  U.S.  Navy  platform.  To  further  narrow 
the  scope  of  our  project,  we  focused  on  18  ships  from  the  U.S.  Navy’s  Ballistic  Missile 
Defense  (BMD)  Fleet.  Five  Ticonderoga-class  guided  missile  cruisers  (CGs)  and 
13  Arleigh  Burke-class  guided  missile  destroyers  (DDGs)  make  up  our  sample.  Critical 
to  the  U.S.  National  Security  Strategy,  the  mission  of  the  BMD  Fleet  is  to  detect,  track, 
and  intercept  ballistic  missiles  of  all  ranges  and  types  (Missile  Defense  Agency,  n.d.). 
The  principal  reason  for  our  selection  of  these  ships  is  not  only  the  importance  of  the 
BMD  mission  but  also  the  commonality  of  systems  contained  within  the  Ticonderoga  and 
Arleigh  Burke  classes.  With  the  Arleigh  Burke  class  still  in  production,  the  results  of  this 
research  will  provide  the  longest  period  of  applicability  and  support  the  largest  class  of 
ships  in  the  naval  inventory. 

C.  PROBLEM  DESCRIPTION 

The  task  of  minimizing  costs  while  maximizing  effectiveness  is  not  one  that  can 
easily  be  achieved  and,  in  some  cases,  may  not  be  realistic.  The  goal  of  this  project, 
however,  is  to  do  just  that  through  an  aggressive  spare -part  allowancing  model  that  will 
improve  the  effectiveness  of  the  U.S.  Navy  surface  fleet  through  a  reduction  in  O&S 
costs.  The  project  analysis  covers  two  distinct  time  periods  to  capture  the  true  value  of 
the  CILS-TAT  and  then  uses  forecasting  models  to  determine  the  future  viability  of  the 
program.  The  first  period  covers  the  years  2003-2009.  During  this  time,  18  CILS-TAT 
reviews  were  conducted  and  48  months  of  operational  data  was  available  per  ship. 
During  the  second  period,  from  2009-2011,  no  CILS-TATs  were  conducted  and  no  other 
allowancing  maintenance  was  done.  Based  on  the  data  available  for  analysis  during  these 
periods,  we  attempted  to  forecast  the  long-term  effectiveness  of  the  CILS-TAT. 
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Unlike  land-based  forces,  where  the  addition  of  another  warehouse  to  store  repair 
parts  is  not  a  major  concern,  on  a  U.S.  Navy  ship,  there  is  a  finite  amount  of  space  to 
store  repair  parts.  In  a  perfect  world,  a  ship  could  carry  a  replacement  part  for  every 
installed  component  in  the  event  that  a  repair  is  required.  Since  that  is  not  feasible,  a 
great  deal  of  consideration  must  be  given  to  detennine  the  right  mix  of  parts  that  make  up 
the  ship’s  onboard  allowance  list.  A  shipboard  allowance  list  for  maintenance  parts  is  the 
larger  list  from  which  spare  items  are  selected,  or  not  selected,  for  onboard  allowances. 

In  the  past,  one  problem  with  such  lists  has  been  assuring  maximum  value 
received  for  dollars  spent  on  spare  items.  Such  value  may  only  be  received  by  making 
sure  that  spare  items  are  ordered  through  some  type  of  a  combinatorial  optimization 
process  (Harrahy,  Powell,  &  Lutz,  1968).  According  to  the  GAO  (2003),  the  U.S. 
Navy’s  spare -parts  supply  problems  can  delay  the  completion  of  needed  maintenance  and 
repair  jobs  on  deployed  ships  and  can  affect  their  operations  and  mission  readiness.  In 
the  same  2003  GAO  report,  covering  two  carrier  strike  groups  over  six  deployments, 
58%  of  the  50,000  maintenance  work  requests  were  delayed  because  the  appropriate  parts 
were  not  located  on  the  ships.  It  can  be  deduced  from  these  studies  that  getting  the  right 
mix  of  parts  onboard  a  ship  is  a  costly,  complex,  and  critical  task  with  far-reaching 
implications.  The  processes  involved  in  allowancing  have  evolved  over  time.  We  cover 
two  such  processes  in  subsequent  chapters. 

Over  the  course  of  several  decades,  initiatives  have  been  implemented  to  either 
improve  the  business  practices  associated  with  allowancing  or  reduce  slow-moving  and 
unnecessary  inventory  (K.  R.  Bitner,  personal  communication,  November  5,  2012). 
Allowances  can  be  reduced  for  a  number  of  reasons  ranging  from  obsolescence  to  a  lack 
of  demand  from  the  system.  When  an  allowance  is  reduced  due  to  a  lack  of  demand, 
greater  risk  is  incurred  in  the  event  of  an  equipment  casualty.  The  severity  of  a  casualty 
can  range  from  an  insignificant  impact  to  the  loss  of  a  critical  system  and  failure  to 
complete  a  primary  mission.  These  circumstances  obviously  cover  a  wide  spectrum  but 
hopefully  provide  some  insight  into  the  factors  that  must  be  considered  at  the  component 
level  when  determining  allowances. 
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One  proposal  to  help  the  U.S.  Navy  more  efficiently  use  its  appropriated  funds 
has  been  to  spend  less  in  the  procurement  of  spare-parts  allowances  onboard  ships.  The 
problem  with  reducing  the  number  of  parts  onboard  is  that  when  a  system  fails  and  the 
parts  are  not  onboard,  logistics  support  is  required — whether  the  part  is  coming  from  the 
other  side  of  the  globe  or  from  a  warehouse  a  few  miles  away.  In  addition  to  the  required 
logistics  support,  the  system  requiring  the  part  is  either  non-operational  or  degraded 
while  awaiting  repairs.  The  costs  of  the  part  not  being  onboard  can  be  measured 
quantitatively  in  monetary  terms  and  more  abstractly  by  a  decrease  in  the  ship’s 
capabilities. 

The  monetary  costs  of  providing  material  support  to  an  operational  vessel  are  the 
most  simple  to  compute  and  typically  involve  determining  the  cost  of  the  part  required, 
the  location  of  the  vessel  requiring  it,  and  the  urgency  of  the  need  for  the  item. 
Transportation  costs  are  reasonably  easy  to  obtain  and  can  be  forecasted  for  a  variety  of 
scenarios  using  simulations.  The  more  difficult  cost  to  capture  is  the  significance  of 
losing  a  mission-critical  system  for  an  extended  period  as  a  result  of  not  having  the  parts 
in  the  right  place  at  the  right  time. 

D.  RESEARCH  OBJECTIVES 

The  objectives  for  our  research  include  the  following: 

1 .  Conduct  a  BCA  of  the  CILS-TAT  process  using  historical  data. 

2.  Measure  the  costs  associated  with  perfonning  a  CILS-TAT. 

3.  Provide  recommendations  to  NSLC  concerning  possible  improvements  to  the 
existing  CILS-TAT  model. 
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E.  RESEARCH  QUESTIONS 

We  have  addressed  the  following  questions  in  our  research: 

1.  How  did  CILS— TAT  impact  allowance  effectiveness  for  our  sample,  and 
at  what  cost? 

2.  Are  there  correlations  between  allowance  effectiveness  rates,  percentage 
of  time  free  (POTF)  from  CASREPs,  and  CILS— TAT  cost? 

3.  What  was  the  effect  of  not  having  CILS— TAT  during  2009-2011,  when 
traditional  allowancing  procedures  were  not  available? 
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II.  BACKGROUND 


A.  ALLOWANCING  FUNDAMENTALS 

When  considering  the  austere  environment  within  which  U.S.  Navy  ships  operate 
and  the  absolute  necessity  to  function  above  readiness  metrics,  it  is  important  that  the 
U.S.  Navy  develop  and  implement  a  robust  set  of  allowancing  procedures  to  ensure  that 
the  right  parts  are  on  the  shelf  when  equipment  fails.  As  related  to  inventory 
management,  allowancing  can  be  defined  as  determining  the  correct  blend  of  items 
carried  onboard  to  ensure  that  the  part  is  available  to  restore  the  equipment  when  failures 
occur.  The  correct  quantity  of  items  is  influenced  by  constraints  such  as  funding  and  the 
cost  of  procurement,  the  availability  of  the  item,  space  availability  onboard  the  ship, 
mean  time  between  failures  (MTBFs),  and  estimated  lead-times  for  replenishment. 

As  mentioned  previously,  the  entire  purpose  of  the  allowancing  process  is  to 
generate  the  right  set  of  parts  to  stock  onboard  the  ship,  thereby  ensuring  that  the  overall 
mission  readiness  levels  of  the  vessel  do  not  drop  below  prescribed  metrics  set  forth  by 
the  U.S.  Navy.  To  measure  the  effectiveness  of  the  allowancing  process,  the  U.S.  Navy 
has  formulated  several  performance  metrics,  such  as  operational  availability  (A0),  supply 
gross  issue  effectiveness,  supply  net  issue  effectiveness,  CASREP  frequency,  and  POTF 
from  CASREPs. 

1.  Operational  Availability 

Operational  Availability  (A0)  provides  a  method  of  predicting  and  assessing 
system  perfonnance  and  readiness  during  the  acquisition  process  and  then  becomes  the 
performance  benchmark  during  initial  operational  capability  (IOC),  deployment,  and 
operations/maintenance  cycles  (Chief  of  Naval  Operations  [CNO],  2003).  The  U.S. 
Navy’s  ability  to  meet  the  highest  readiness  levels  is  principally  derived  by  the  A0  of  the 
warfighting  systems  and  equipment  installed  onboard  U.S.  Navy  vessels. 

According  to  Chief  of  Naval  Operations  Instruction  (OPNAVINST)  3000. 12A 
(CNO,  2003),  the  calculation  for  determining  the  A0  is  a  probability  function  of 
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reliability,  maintainability,  and  supportability  components.  The  equation  for  determining 
A0  is  written  as  the  following:  system  up  time  divided  by  total  time  (up  time  +  down 
time). 

Operational  Availability  is  fundamentally  considered  as  a  supportability 
calculation  of  the  equipment/system  in  terms  of  predicted  reliability  (R),  called  mean 
time  between  failure  (MTBF);  predicted  maintainability  (M),  in  terms  of  mean  time  to 
repair  (MTTR);  and  designed  supportability,  called  mean  logistics  delay  time  (MLDT; 
CNO,  2003).  Inherent  availability  (A;),  an  alternative  metric,  does  not  include  a 
consideration  of  support  functions  related  to  re-supply  transportation  and  repair.  Once  A; 
is  determined,  you  are  ready  to  add  in  the  supportability  portion,  or  MLDT,  to  calculate 
A0  (CNO,  2003).  See  Figure  3  for  a  graphical  representation  of  A;  and  A0. 


A;  = 

MTBF 

MTBF  +  MTTR 

MTBF 

A0  = 

MTBF  +  MTTR 

+  MLDT 

Hardware/Software 
Design  Considerations 

Logistics  System 
Design  Considerations 

Figure  3.  Logistics  Impact  on  Operational  Availability 

(From  CNO,  2003) 


2.  Measuring  Allowancing  Effectiveness 

In  order  to  effectively  measure  the  allowancing  packages  that  the  U.S.  Navy  is 
developing  and  sending  to  the  fleet,  they  have  constructed  several  calculations  to 
interpret  the  demand  signal  against  the  assets  onboard.  The  metrics  are  assembled  by 
segregating  the  demand  into  different  pools  or  supply  source  codes  based  on  the  stock 
posture  of  the  requirement.  Figure  4  represents  the  various  codes. 
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CODE  DEFINITION 

A  Allowance  List  Material  issued  from  storeroom  stock 
C  Non- Allowance  List  Material  (SIM)  issued  from  storeroom  stock 
D  Allowance  List  Material  Not  In  Stock  (NIS)  when  requested 

Non- Allowance  List  Material  (SIM)  Not  In  Stock  (NIS)  when 
requested 

Not  Carried  (NC)  repair  parts  which  are  not  listed  on  an  APL  in  the 
ship's  COS  file 

Not  Carried  (NC)  repair  parts  which  are  listed  on  an  APL,  but 
does  not  compute  for  an  allowance 

Figure  4.  U.S.  Navy  Allowance  Supply  Source  Codes 

(From  Naval  Supply  Systems  Command  [NAVSUP],  1997) 

The  calculation  used  in  evaluating  allowance  performance  is  the  allowance 
effectiveness  metric,  which  is  represented  by  the  following  equation:  (A  +  C  +  D  + 
F)/(A  +  C  +  D  +  F  +  G+  J ).  The  allowance  effectiveness  calculation  allows 

stakeholders  to  determine  the  strength  of  the  allowanced  items  as  well  as  the  non¬ 
allowance  material  selective  item  maintenance  (SIM)  against  demand  (NAVSUP,  1997). 

3.  Supply  Issue  Effectiveness  Rates 

In  an  effort  to  measure  the  overall  issue  effectiveness  of  the  allowancing  packages 
throughout  the  training  cycle  and  into  deployment,  the  U.S.  Navy  has  developed 
a  set  of  inventory  issue  effectiveness  rates  to  judge  the  ability  of  the  inventory  to 
meet  the  demands  placed  against  that  inventory.  There  are  two  metrics  that  are  used  to 
determine  the  success  of  the  allowancing  package,  the  first  being  the  supply  gross  issue 
effectiveness  rate.  Supply  gross  issue  effectiveness  measures  the  number  of  requirements, 
commonly  called  requisitions,  issued  against  the  total  number  of  requirements.  Supply 
gross  issue  effectiveness  is  computed  by  dividing  the  number  of  requisitions  issued  by 
the  total  number  of  requisitions:  (A  +  C)/(A  +  C  +  D  +  F  +  G  +  J). 

The  aforementioned  performance  measurement  allows  the  managers  of  the 
allowancing  package  to  quantitatively  judge  whether  or  not  they  have  the  correct  variety 
and  quantity  of  parts  on  the  ship.  The  calculation  provides  a  percentage  value  for  every 
demand  that  was  issued  from  the  storeroom.  In  other  words,  it  reveals  how  many  parts 
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were  not  onboard  that  should  have  been  onboard  given  the  current  demand.  The  output 
from  this  calculation  would  provide  an  allowancing  package  stakeholder  with  a  sense  of 
how  many  parts  or  line  items  must  be  added  to  the  allowancing  package  to  meet  issue 
effectiveness  goals. 

The  second  metric  that  the  U.S.  Navy  uses  is  the  supply  net  issue  effectiveness 
rate.  This  measurement  tool  is  calculated  by  dividing  the  total  number  of  issued 
requisitions  by  the  total  number  of  requisitions  minus  the  number  of  requisitions  not 
carried  (NC)  onboard:  (A  +  C)/(A  +  C  +  D  +  F).  This  more  focused  rate  allows  the 
stakeholders  to  obtain  a  sense  of  how  the  variety  and  quantity  of  the  parts  included  in  the 
allowancing  package  are  reacting  to  the  current  demand  signal.  For  the  purpose  of 
allowancing  package  development,  the  use  of  the  supply  gross  issue  effectiveness  rate 
would  prove  to  be  more  useful  since  stakeholders  desire  to  know  on  average  how  well  the 
allowancing  package  supported  the  demand  as  a  whole. 

4.  Casualty  Reporting 

A  measure  of  effectiveness  central  to  our  project  involves  tracking  the  number 
and  severity  of  CASREPs  for  each  of  the  18  ships  in  the  sample.  A  CASREP  is  a  report 
made  to  the  ship’s  operational  chain  of  command  concerning  a  significant  equipment 
malfunction  that  cannot  be  corrected  within  a  48-hour  period.  The  release  of  a  CASREP 
will  also  alert  supporting  activities  of  the  urgency  to  provide  aid  in  the  form  of  spare  parts 
or  technical  assistance.  The  significance  of  each  CASREP  is  determined  by  the  impact  of 
the  equipment  failure  on  the  ship’s  mission.  There  are  different  categories  of  CASREPs, 
ranging  from  Category  Two  (C2)  through  Category  Four  (C4),  with  C4  being  the  most 
severe.  A  C4  CASREP  denotes  that  a  deficiency  exists  in  mission-essential  equipment 
that  causes  the  loss  of  at  least  one  primary  mission.  A  C3  CASREP  denotes  that  a 
deficiency  exists  in  mission-essential  equipment  that  causes  a  major  degradation  but  not 
the  loss  of  a  primary  mission.  Finally,  a  C2  CASREP  indicates  that  the  ship  has  lost 
redundancy  in  one  of  its  primary  or  secondary  mission  areas.  CASREPs  are  required  to 
be  updated  periodically  by  the  ship  until  the  damaged  equipment  is  restored  to  a  fully 
mission-capable  state.  The  frequency  of  updates  is  correlated  to  the  severity  of  the 
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CASREP.  The  CASREP  will  remain  an  open  report  until  all  repairs  are  finalized 
(NAVSUP,  1997). 

5.  Percentage  of  Time  Free  From  CASREPs 

POTF  is  a  readiness  metric  tracked  by  NAVSUP  Corporate  Information  Systems 
(CIS)  that  records  the  number  of  days  in  a  reporting  period  that  a  ship  does  not  have  an 
open  C3  or  C4  CASREP.  This  metric  is  a  central  measure  of  a  ship’s  material  readiness 
and  one  that  we  use  in  the  analysis  section  of  this  report.  A  calculation  of  time  free  of 
casualties  is 
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where 


i  =  a  running  index  for  individual  ships  in  the  grouping  that  have  been 
active  for  more  than  1/5  of  the  calendar  period  and 

K  =  total  ships  in  the  grouping  meeting  the  active  time  criteria. 

B.  ALLOWANCING  STAKEHOLDERS 

The  entire  allowancing  process  is  completed  by  a  vast  array  of  logistics  and 
maintenance  professionals  employed  by  the  U.S.  Navy  and  civilian  partners.  The 
requirements  necessary  to  develop  a  comprehensive  allowancing  package  deem  that  all 
influencers  collaborate  effectively  as  one  cohesive  team.  The  requirements  for  the 
allowancing  package  range  from  ship  configuration  data,  maintenance  and  logistics  data, 
and  if  available,  logistics  support  data  specifically  related  to  reliability  concerns. 

On  the  logistics  side  of  the  house,  there  are  a  couple  of  key  players  across  the 
U.S.  Navy.  The  first  of  these  players — and  probably  the  most  dominant  in  calculating  the 
allowances — is  the  NAVSUP  Weapons  System  Support,  Mechanicsburg  (WSS  Mech). 
They  are  the  U.S.  Navy’s  representative  to  manage  the  allocation  of  resources  relating  to 
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the  allowancing  process.  NAVSUP  WSS  Mech  possesses  the  models  necessary  to 
develop  allowancing  packages  and  work  with  all  stakeholders  to  ensure  that  the  end 
product  fits  the  needs  of  the  warfighter.  In  addition,  they  are  tasked  with  controlling  the 
funding  for  initial  outfitting  onboard  the  vessels. 

On  the  maintenance  side  of  the  house,  the  stakeholder  most  concerned  with  the 
development  of  the  allowancing  process  is  the  Naval  Sea  Systems  Command 
(NAVSEA).  They  are  tasked  with  ensuring  that  the  ship  is  configured  properly  with  the 
most  up-to-date  systems  and  equipment.  To  ensure  that  the  configuration  of  all  systems 
is  correct,  NAVSEA  works  closely  with  NSLC,  who  provides  integrated  logistics 
support.  The  configuration  of  the  ship  must  be  accurate  when  developing  the 
allowancing  package  to  facilitate  superior  sustainability  during  the  entire  life  cycle  of  the 
ship. 

When  developing  a  product,  you  must  have  a  customer.  In  this  case,  the  customer 
is  the  warfighter  on  the  waterfront:  Commander,  Fleet  Forces  Command  (CFFC).  CFFC 
is  charged  by  the  CNO  to  ensure  that  all  fleet  forces  maintain  the  highest  levels  of 
operational  readiness  to  support  the  nation’s  maritime  strategy.  With  regard  to  surface 
ships,  Commander,  Naval  Surface  Forces  (SURFOR)  is  the  stakeholder  involved  in  the 
allowancing  process  and  subsequent  reviews  of  that  process  as  the  ship  progresses 
throughout  its  life  cycle.  Within  SURFOR,  there  are  maintenance  and  logistics 
components  that  work  in  concert  with  their  counterparts  involved  in  the  process. 
Fundamentally,  once  SURFOR  takes  ownership  of  the  vessel,  it  is  ultimately  responsible 
if  the  ship  does  not  meet  the  mission. 

The  maintenance  personnel  at  SURFOR  manage  the  maintenance  practices  and 
configuration  changes  related  to  keeping  the  ship  up  to  date.  The  logisticians  are 
constantly  monitoring  the  issue  effectiveness  rates  to  determine  if  an  allowancing 
package  should  be  applied  to  increase  the  mission  readiness. 

C.  FUNDING 

The  allowancing  process  requires  two  different  classifications  of  funding  to  pay 
for  all  inventory  required:  initial  issue  provisioning  and  follow-on  replenishment  over  the 
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life  cycle  of  the  allowance  being  tied  to  that  vessel.  The  reasoning  behind  using  two 
different  classifications  of  money  is  the  delineation  between  the  Budget  Activities  (BA) 
of  Operations  and  Maintenance,  Navy  (0&M,N)  and  Other  Procurement,  Navy  (OPN) 
within  the  DoD  budget.  The  two  classifications  of  money  are  controlled  by  separate 
organizations  within  the  U.S.  Navy. 

The  funding  stream  more  closely  tied  to  the  allowancing  process  and  initial 
provisioning  is  the  OPN  appropriation.  This  appropriation  is  designated  for  the 
procurement,  production,  and  modernization  of  support  equipment  and  materials  not 
otherwise  provided  for  as  well  as  the  procurement  and  installation  of  equipment 
(Department  of  the  Navy  [DoN],  2012).  Because  of  the  designation  of  these  funds  to 
support  the  initial  outfitting  of  naval  forces,  all  initial  provisioning  within  an  allowancing 
package  are  paid  for  with  OPN  funds. 

The  allocation  of  these  funds  from  Congress  is  given  to  NAVSUP  WSS  Mech  to 
determine  the  best  allocation  across  the  competing  priorities  within  the  enterprise.  The 
funds  are  then  designated  as  OPN-8  funds.  The  Navy  Working  Capital  Fund  (NWCF) 
then  buys  spare  parts  listed  in  the  new  allowancing  product  with  OPN-8  funds  following 
the  post-material  support  date  (MSD;  CNO,  2012).  Once  the  allowancing  products  are 
developed  with  the  inputs  from  stakeholders,  NAVSUP  WSS  Mech  makes  the 
determination  to  purchase  the  requisite  spare  parts  to  fill  the  allowances. 

The  appropriations  assigned  to  the  BA  of  0&M,N  are  designated  to  finance  the 
day-to-day  costs  of  operating  naval  forces,  including  fuel,  supplies,  and  maintenance  of 
ships,  U.S.  Navy  and  Marine  Corps  aircraft,  related  weapons  systems,  and  the  support 
establishment  ashore  (DoN,  2012).  Consequently,  any  replenishment  for  stock  that  the 
ship  might  require  to  maintain  operational  readiness  must  be  paid  for  with  0&M,N 
dollars.  At  the  force  level,  these  funds  are  controlled  by  the  type  commander  (TYCOM) 
and  allocated  down  to  the  unit  supply  officer  on  a  quarterly  basis  in  the  form  of  an 
operating  target  (OPTAR).  The  OPTAR  will  pay  for  all  spare-part  requirements  and  any 
related  costs,  to  include  transportation  costs,  ordering  costs,  and  any  holding  costs 
associated  with  keeping  the  part  in  inventory. 
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D.  ALLOWANCING  PRODUCT  MAKEUP 


1.  Depot-Level  Repairable  Versus  Non-Depot-Level  Repairable 

The  typical  allowancing  package  is  composed  of  a  variety  of  different  line  items 
and  quantities  dependent  on  the  type  of  equipment  onboard  the  ship,  the  dollar  value,  the 
required  endurance  level,  and  the  average  MTBF.  The  most  basic  classification  among 
the  varying  spare  parts  in  the  allowancing  package  is  the  designation  as  a  depot-level 
repairable  (DLR)  item  versus  a  non-DLR  item,  otherwise  known  as  a  consumable  item. 

DLR  items  are  usually  the  high-dollar-value  components  and  are  deemed  too 
costly  to  dispose  of.  This  classification  requires  that  when  a  DLR  is  unserviceable,  it 
must  be  sent  back  to  the  pre-detennined  repair  facility  for  overhaul  and  subsequent 
clearance  as  ready  for  issue  (RFI).  When  considering  the  dollar  value  of  these  items,  the 
U.S.  Navy  aggressively  tracks  the  movement  and  repair  status  of  all  items  both  on  the 
vessels  and  ashore  to  eliminate  unnecessary  waste.  As  a  rule  of  thumb,  the  allowance  for 
a  DLR  in  an  allowancing  package  is  incapable  of  being  altered  by  the  personnel  onboard 
the  ship.  These  allowances  are  to  remain  as  they  were  when  they  were  implemented. 
The  designation  of  DLRs  is  detennined  by  NAVSUP  guidance  (NAVSUP,  1997). 
Specific  codes  are  assigned  to  segregate  the  DLRs  by  their  application  of  use  in  a 
particular  system  and  into  manageable  groupings  based  on  their  level  of  usage.  In 
addition,  these  codes  are  used  to  designate  which  inventory  manager  will  handle  the  day- 
to-day  administration  of  that  particular  item. 

The  non-DLR  items  are  not  tracked  as  closely  but  still  require  a  compulsory  level 
of  oversight.  Consumable  items  are  kept  onboard  the  ship  to  be  used  as  necessary;  once 
the  life  of  that  asset  has  been  exhausted,  the  item  is  simply  discarded  in  accordance  with 
published  guidelines.  The  logistics  personnel  onboard  the  ships  are  given  authorization 
to  manipulate  the  allowance  of  these  items  as  they  see  fit  to  meet  future  demand  signals. 
The  following  is  a  list  of  the  most  commonly  found  allowance  classifications  for  non- 
DLR  items:  9C,  9B,  9G,  9N,  and  9E  (NAVSUP,  1997). 
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2. 


Allowance  Type  Codes 


When  apportioning  allowances,  the  allowancing  package  will  assign  a 
classification  code  to  the  assets  within  the  package.  This  will  enable  the  logistics 
personnel  to  know  by  which  process  the  asset  was  placed  on  the  ship.  The  classification 
codes  are  broken  down  into  nine  different  categories,  called  allowance  type  (AT)  codes. 
The  most  common  classification  with  regard  to  allowancing  products  is  an  ATI.  This 
classification  explains  that  the  asset  is  mandated  to  be  carried  because  of  the 
implementation  of  an  allowancing  product.  A  breakdown  of  the  other  applicable  AT 
codes  and  descriptions  are  captured  in 


Table  1.  Allowance  Type  Codes  (From  NAYSUP,  1997) 


AT  Code 

Name 

Description 

1 

Cosal  Item 

COSAL  item. 

2 

Aviation  Support  Item 

Load  List  item,  applicable  to  load  carrying  ships  only. 

3 

COSAL/AV  Item 

Load  and  Allowance  List  item,  applicable  to  load  carrying  ships  only. 

4 

Demand  Based  Item  (DBI) 

Non-COSAL  item  that  is  stocked  based  solely  on  demand. 

5 

TYCOM  Directed  Item 

Non-COSAL  authorized  add  item  based  on  specific  TYCOM  authority. 

6 

Excess  Item 

Non-COSAL  excess  item  that  does  not  have  sufficient  demand  to  maintain.  The 

item  is  to  be  offloaded  and  deleted. 

7 

Economic  Retention  Item 

Economic  retention.  Excess  item  that,  because  of  low  unit  cost  (normally  under 
$100.00),  is  authorized  for  retention  until  the  next  ILO/ReAVCAL.  The  exact 
amount  may  be  specified  by  each  TYCOM. 

8 

Demand  Recording 

Non-COSAL  item  that  is  established  for  demand  recording  only.  It  will  be  changed 
to  AT  code  4  if  demand  reaches  established  criteria,  otherwise  it  will  be  deleted 
after  24  months  with  no  demand. 

9 

Substitute/Alternate  Item 

Assigned  to  a  substitute  item  which  is  not  stocked  as  a  primary  number. 

3.  Mission  Criticality  Codes 

The  U.S.  Navy’s  Item  Mission  Essentiality  Coding  (IMEC)  system  consists  of  a 
combination  of  military  essentiality  code  (MEC)  and  mission  criticality  code  (MCC). 
These  codes  help  the  allowancing  product  developers  to  apply  weights  to  those  assets  that 
are  most  important  to  the  sustainability  and  readiness  of  the  ship  within  the  allowancing 
model.  The  allowancing  package  will  contain  either  the  MEC  or  MCC  in  Coordinated 
Shipboard  Allowance  List  (COSAL)  Part  I,  Sections  A  and  B,  depending  on  the  ship’s 
computation  method  (DoN,  2009).  The  possible  MCCs  that  can  be  assigned  to 
components  are  listed  in  Table  2. 
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Table  2.  Mission  Criticality  Codes  (From  DoN,  2009) 

CODE  DEFINITION 

1  Failure  of  component/equipment  causes  minor  mission  impact. 

^  Failure  of  component/equipment  causes  total  loss  or  severe 
degradation  of  a  secondary  mission. 

^  Failure  of  component/equipment  causes  severe  degradation  of  a 
primary  mission  capability. 

Failure  of  component/equipment  causes  total  loss  or  severe 
4  degradation  of  mobility  or  primary  mission  (propulsion  or  life 

support). 

The  loss  of  this  equipment  results  in  a  safety  hazard  to  the  ship  or  its 
crew. 

X  Assigned  to  all  Allowance  Equipage  Lists  (AEL). 


E.  ALLOWANCING  INPUT  AND  OUTPUT 

1.  Inputs 

The  process  for  developing  an  allowance  package  requires  several  pieces  of  data 
from  varying  stakeholders  within  the  process.  The  information  necessary  to  effectively 
determine  the  allowancing  package  ranges  from  configuration  management  data  to 
maintenance  history  data  located  within  the  ship’s  Maintenance,  Material  and 
Management  (3M)  database. 

The  configuration  management  data  is  taken  from  the  Configuration  Data 
Managers  Database-Open  Architecture  (CDMD-OA),  which  is  managed  by  NSLC. 
CDMD-OA  tracks  the  status  and  maintenance  of  naval  equipment  and  their  related 
logistics  items  (e.g.,  drawings,  manuals)  on  ships  and  naval  activities  around  the  world. 
The  status  of  a  given  piece  of  equipment  on  a  ship  determines  what  and  how  many  spare 
parts  will  be  stored  on  that  ship  for  that  equipment,  making  this  tracking  extremely 
important  in  terms  of  cost,  shipboard  space  and  weight,  and  the  operational  availability  of 
the  ship  (CDMD-OA,  2012).  The  U.S.  Navy  identifies  the  required  spare  parts  onboard 
U.S.  Navy  vessels  through  the  use  of  allowance  parts  lists  (APLs).  APLs  provide  support 
and  outfitting  for  parts  that  are  required  for  the  particular  maintenance  action  performed 
onboard  the  ship  (Alvarez,  2010). 
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The  3M  database  is  a  central  point  on  the  ship  where  all  data  is  kept  related  to 
maintenance  performed.  In  addition,  the  database  contains  a  comprehensive  list  of  all 
assets  that  are  and  should  be  kept  on  the  ship  to  perfonn  maintenance.  Along  with  the 
comprehensive  list  is  the  associated  usage  data  for  the  parts.  If  the  ship  is  a  new 
construction,  the  usage  data  from  similar  vessels  across  the  fleet  will  be  used  in  the 
development  of  the  allowancing  package  for  the  new  vessel. 

In  addition  to  gathering  the  maintenance,  demand,  and  configuration  data  from 
the  3M  system,  data  must  also  be  extracted  from  the  ship’s  current  stock  record  file 
(SRF).  The  SRF  contains  the  financial  records  of  the  ship  to  gauge  the  impacts  of  the 
previous  allowancing  product  against  the  current  funding  levels.  This  information  can  be 
very  telling  as  to  whether  or  not  the  previous  product  was  effective  at  saving  costly 
procurements  and  the  inclusion  of  unnecessary  carrying  costs. 

The  final  portion  of  information  that  must  be  incorporated  into  the  allowancing 
package  is  the  Weapons  System  File  (WSF)  Level  C,  which  contains  all  provisioning  and 
technical  decisions. 

2.  Outputs 

With  the  consideration  of  the  aforementioned  input  of  configuration,  financial, 
maintenance,  and  demand  data,  the  stakeholders  are  capable  of  applying  this  infonnation 
into  a  sophisticated  mathematical  allowance  model  that  provides  the  optimized 
allowancing  package.  The  model  can  be  manipulated  to  varying  degrees,  pennitting 
different  types  of  allowancing  packages  dependent  on  the  desires  of  the  stakeholders.  In 
the  next  section,  we  explore  several  different  types  of  allowancing  packages. 

Once  the  model  produces  the  output,  the  package  is  socialized  among  the 
stakeholders.  Once  it  is  finalized,  the  allowances  are  generated  by  NAVSUP  WSS  Mech 
and  funded  and  released  to  SURFOR  for  issuance  to  the  ship.  The  final  piece  of  the 
puzzle  is  that  the  ship  will  integrate  the  new  product  into  the  ship’s  database  and  drop  the 
order  requisitions  to  stock  the  shelves  accordingly. 
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F.  TYPES  OF  ALLOWANCING  PACKAGES 

1.  Comprehensive  Allowancing 

As  previously  discussed,  the  output  product  from  the  allowancing  process  can  be 
tailored  to  focus  on  a  particular  weapons  system  or  set  to  review  the  entire  ship’s  spare 
parts  support  package.  The  most  common  comprehensive  allowancing  package  is  the 
COSAL.  The  COSAL  provides  both  technical  and  supply  information,  which  makes  it  an 
integrated  logistics  support  (ILS)  document.  It  is  a  technical  document  to  the  extent  that 
equipment/component/part  nomenclatures,  operating  characteristics,  technical  manuals, 
etc.  are  described  in  APLs  or  allowance  equipage  lists  (AELs;  NAVSUP,  1997). 

Implementation  of  the  COSAL  is  typically  conducted  during  the  initial  outfitting 
of  the  platform  at  the  beginning  of  its  life  cycle.  Factors  determining  the  composition  of 
the  COSAL  are  the  maintenance  philosophy/capability,  support  concept,  logistics 
response  time,  historical  demand,  and  item/system  population  and  redundancy.  Since  the 
COSAL  is  the  primary  listing  for  all  spare  parts  allocated  for  the  ship,  logisticians 
routinely  reference  the  COSAL  in  their  day-to-day  operations.  Purportedly,  the  COSAL 
is  designed  to  provide  the  warship  with  a  sustained  level  of  material  support  for  90 
combat  days  without  replenishment  and  is  tailored  to  a  particular  ship  class,  because  each 
ship  class  has  a  different  weapons  system  configuration  (Axinto  &  Giles,  2005). 

Once  implemented  onboard  the  ship,  the  COSAL  begins  to  receive  periodic 
updates  to  support  any  configuration  or  allowancing  modifications.  Because  COSALs 
are  costly  to  implement  and  very  time  intensive,  the  U.S.  Navy  has  developed  a  system  of 
implementing  the  incremental  changes  to  COSALs  by  way  of  an  automated  shore 
interface  (ASI).  These  updates  are  specifically  tailored  to  a  particular  ship  or  class  to 
ensure  that  the  COSAL  remains  up  to  date  with  configuration  changes  or  additions  and 
deletions  to  the  allowancing  package.  The  periodicity  of  ASI  updates  varies  depending 
on  the  timing  of  the  changes  included  in  the  updates  and  the  funding  levels  available  to 
support  those  changes.  The  process  of  using  ASIs  is  far  less  costly  and  less  labor 
intensive  on  the  ship  and  the  shore  facilities. 
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2.  Targeted  Allowancing 

Considering  the  significant  costs  involved  with  developing  and  implementing  a 
COSAL,  the  U.S.  Navy  has  worked  towards  minimizing  those  costs  by  employing  a 
targeted  allowancing  process.  Stakeholders  use  this  focused  approach  to  zero  in  on  the 
spare-parts  requirements  that  would  provide  the  greatest  benefit  towards  increased 
readiness  of  the  shipboard  allowancing  package.  The  process  of  targeted  allowancing 
follows  a  similar  path  to  the  comprehensive  review,  with  one  exception.  A  filter  is 
applied  to  the  mathematical  model  that  only  lets  certain  aspects  of  the  COSAL  be 
reviewed  based  on  criteria  set  forth  by  the  stakeholders.  Once  the  targeted  allowancing 
package  is  produced,  the  implementation  process  is  identical  and  the  funding  follows  a 
similar  stream  for  procurement  of  the  assets. 

G.  CONTINUOUS  INTEGRATED  LOGISTICS  SUPPORT-TARGETED 

ALLOWANCE  TECHNIQUE  (CILS-TAT) 

1.  Overview 

One  of  the  common  targeted  allowancing  techniques  that  the  U.S.  Navy  has 
adopted  is  the  process  tenned  the  CILS-TAT.  A  CILS-TAT  is  used  to  focus  the 
allowancing  process  to  minimize  the  overall  funding  requirement,  increase  support  for 
poor-performing  equipment,  and  reduce  allowance  churn  while  providing  the  greatest 
increase  to  readiness  for  the  ship.  In  the  current  fiscally  constrained  environment,  this 
approach  is  absolutely  vital  to  ensuring  that  the  dollars  are  spent  optimally. 

The  U.S.  Navy’s  Maritime  Allowance  Working  Group  (MAWG)  began 
discussing  the  use  of  the  CILS-TAT  in  1999  with  15  prototype  ships.  The  concept  was 
to  develop  a  technique  by  which  ship-optimized  allowances  were  discreetly  applied  to 
update  a  ship’s  SRF,  eliminating  allowances  with  no  usage  and  targeting  systems  with 
usage  for  allowance  updates  of  either  range  additions  or  depth  increases.  During  the 
prototype  phase,  the  MAWG  measured  success  by  looking  at  the  increase  in  supply 
effectiveness,  reduction  in  cost,  and  churn  of  the  allowances.  Through  evaluation  of 
these  metrics  specifically,  they  recognized  a  47%  and  50%  reduction  in  the  line-item 
chum  and  cost  of  new  allowances,  respectively,  during  the  prototype  phase  (Bruno, 
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1999).  The  results  of  the  prototypes  also  revealed  a  0.9%  increase  in  the  supply  gross 
issue  effectiveness,  from  51.8%  to  52.7%  (Bruno,  1999). 

From  1999  to  2009,  the  U.S.  Navy  adopted  CILS-TAT  as  one  of  the  primary 
methods  for  conducting  targeted  allowancing  and  has  implemented  the  process  on  many 
different  platforms,  to  include  Los  Angeles-class  fast-attack  submarines  (SSNs),  Arleigh 
Burke-class  DDGs,  Avenger-class  mine  countermeasures  ships  (MCMs),  Oliver  Hazard 
Perry-class  frigates  (FFGs),  and  Ticonderoga-class  cruisers  (CGs).  The  selection  process 
for  implementation  of  a  CILS-TAT  on  a  ship  is  conducted  by  various  stakeholders 
included  in  the  MAWG,  to  include  representatives  from  NSLC,  NAVSUP  WSS  Mech, 
NAVSEA,  and  the  TYCOM. 

2.  Time  Line 

The  time  line  for  implementation  typically  follows  the  deployment  cycles  and  is 
displayed  in  Figure  5.  After  a  ship  has  completed  two  years  of  normal  underway 
operations,  data  is  collected  from  that  period  and  evaluated  by  the  MAWG  to  detennine 
if  the  ship  is  a  candidate  for  a  CILS-TAT.  As  they  assess  the  data,  several  key  factors 
influence  the  decision  made  by  the  MAWG,  ranging  from  the  availability  of  time  for 
implementation  (i.e.,  maintenance  availability  opportunities),  a  recognized  reduction  of 
the  supply  gross  issue  effectiveness,  a  decrease  in  readiness  levels  below  the  fleet 
average,  and  funding  availability. 


Figure  5.  CILS-TAT  Implementation  Time  Line 
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Once  the  CILS-TAT  is  completed  and  ready  for  implementation,  the  product  is 
given  to  the  TYCOM,  who  is  then  assigned  the  responsibility  of  working  with  the 
operational  unit  to  implement  the  product  into  the  ship’s  onboard  databases.  In  an  effort 
of  continuous  evaluation,  at  the  conclusion  of  another  two-year  period,  the  MAWG  will 
once  again  review  the  health  of  the  ship  and  determine  if  the  process  was  profitable  and  if 
another  CILS-TAT  product  should  be  introduced. 

3.  Input 

In  order  to  develop  the  CILS-TAT,  product  stakeholders  draw  from  several 
different  systems  and  add  a  filter  into  the  process  to  focus  on  a  narrowly  defined  set  of 
data.  The  systems  used  in  the  CILS-TAT  are  the  same  as  those  used  to  formulate  a 
comprehensive  allowancing  product  and  include  the  CDMD-OA,  3M  data  from  the  ship, 
CASREP  information,  and  the  WSF.  Once  this  information  is  compiled,  it  is  run  through 
an  elaborate  mathematical  allowancing  model.  The  final  piece  in  the  application  is  a 
filter  that  focuses  the  allowancing  package  to  only  look  at  a  particular  type  of  3M  data. 
Once  the  filter  is  applied,  the  output  file  is  generated  and  reviewed.  The  CILS-TAT’s 
filter  only  (a)  allows  new  allowances  to  APLs  that  have  had  3M  usage,  (b)  provides 
allowance  deletions  or  decreases  for  those  APLs  with  no  reported  usage,  or  (c)  provides 
deletions  for  parts  that  no  longer  support  installed  configurations.  This  filter  is  what 
makes  this  allowancing  product  a  more  targeted  approach. 

4.  Output 

The  output  provided  by  the  model  is  an  allowancing  package  that  truly  focuses  on 
improving  the  collection  of  parts  necessary  to  support  the  ship  based  on  the  demand  that 
has  been  realized  over  the  previous  two  years.  Those  systems  that  have  needed  a 
particular  asset  onboard  but  the  part  was  not  available  would  now  be  added  to  the  ship’s 
inventory,  thereby  improving  the  supply  gross  issue  effectiveness  and  operational 
readiness.  Specifically,  the  output  file  would  include  (a)  range  additions  to  prevent 
missing  a  future  demand  and  (b)  range  and  depth  decreases.  Those  parts  that  historically 
have  not  been  demanded  or  do  not  currently  confonn  to  the  current  configuration 
onboard  would  be  offloaded  through  attrition.  This  more  focused  approach  truly  reduces 
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the  cost  of  allowancing,  especially  when  you  calculate  the  cost  of  allowancing  over  the 
entire  U.S.  Navy  fleet  while  providing  the  most  efficient  use  of  dollars  to  improve  overall 
operational  readiness. 

5.  CILS-TAT  Process  Flow 

The  actual  process  of  a  CILS-TAT  begins  with  the  collection  of  24  months  of 
operational  data  (see  Figure  5).  The  historical  demand  data  is  then  added  to  the  ship’s 
current  COSAL  and  analyzed  for  allowance  additions  and  decreases.  CILS-TATs  are 
ship  specific,  meaning  that  no  two  ships  will  experience  exactly  the  same  types  or 
frequency  of  material  demands.  There  is  a  degree  of  commonality  among  demands,  but 
due  to  different  missions  performed,  environmental  factors  experienced  in  the  area  of 
operation,  and  the  overall  material  condition  of  the  ship,  these  CILS-TATs  have  to  be 
performed  on  an  individual  basis. 

Once  the  24  months  of  data  is  drawn  down  from  the  ship,  the  demands  registered 
for  scheduled  and  unscheduled  maintenance  are  analyzed.  A  CILS-TAT  will  also  add 
new  allowances  when  a  system  configuration  change  is  in  process  or  expected  to  happen 
in  the  near  future.  Allowances  will  also  be  added  to  the  SRF  in  anticipation  of  required 
maintenance  for  each  of  those  new  systems. 

Once  analysis  of  both  the  historical  demand  data  and  forecasted  equipment 
upgrades  are  complete,  the  new  allowances  and  the  remainder  of  the  COSAL  product  is 
sent  to  the  ship  for  loading  into  the  central  database.  The  CILS-TAT  output  is  then 
validated  by  comparing  the  total  list  of  national  item  identification  numbers  (NIINs) 
recommended  by  the  CILS-TAT  against  the  allowances  that  the  ship  already  has 
onboard.  Any  shortfalls  identified  during  this  validation  become  candidates  for 
procurement.  To  simplify  this  with  an  example,  a  CILS-TAT  may  recommend  that  the 
ship  carry  5,000  different  repair  parts  at  a  cost  of  $2.5  million  but,  after  validating  the 
CILS-TAT  against  the  ship’s  current  onboard  allowances,  determines  that  only  300  new 
parts  at  a  cost  of  $250,000  may  be  additionally  required.  A  CILS-TAT  not  only 
recommends  new  allowances  but  also  allows  the  ship  to  validate  its  current  configuration. 
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Once  all  shortfalls  are  identified,  requisitions  are  prepared  for  each  new  NUN  to 
be  procured.  The  ship  is  not  required  to  prioritize  the  new  allowance  requisitions  since 
all  additional  allowances  are  considered  valid  by  NSLC  and  will  be  further  evaluated  by 
the  supply  system  during  the  requisition  filling  process.  There  are  three  distinct  phases  of 
further  evaluation  that  each  requisition  goes  through  before  funding  is  obligated.  We 
cover  these  phases  in  the  next  section. 

6.  Requisition  Screening 

Once  requisitions  are  released  into  the  supply  system  as  a  result  of  a  CILS-TAT, 
they  are  reviewed  by  the  NAVSUP  Outfitting  Support  Activity  (OS A)  and  Outfitting 
Requisition  Control  Accounting  System  (ORCAS)  managers.  The  purpose  of  this 
screening  is  to  conduct  quality  checks  and  ensure  that  the  correct  fund  codes  and  advice 
codes  were  applied  to  each  requisition  by  the  ship.  These  codes  play  an  important  role  in 
determining  the  right  type  of  funding  applied  to  each  of  the  requisitions  and  confirming 
that  the  obligations  and  expenditures  eventually  match  for  each.  Requisitions  flow  freely 
through  this  process  and  are  not  held  up  for  funding  constraints.  The  main  concern  here 
is  to  ensure  that  the  requisition  was  submitted  properly  by  the  ship. 

Once  the  requisitions  complete  the  quality  assurance  process,  they  then  move  to 
the  Pushed  Material  Inventory  Control  System  (PMICS)  offices,  which  are  a  branch 
within  NSLC.  The  new  allowances  recommended  by  the  CILS-TAT  are  compared 
against  allowances  that  are  already  being  procured  via  other  means — for  example,  the  In- 
Service  Engineering  Activity  (ISEA).  A  contractor  may  already  have  the  responsibility 
to  provide  initial  outfitting  support  for  a  particular  system  or  group  of  systems  onboard 
the  ship.  The  screen  through  PMICS  is  done  to  look  at  all  possible  avenues  from  which 
this  support  could  come  and  ensure  that  the  government  is  not  paying  for  the  same  initial 
allowance  for  the  same  ship  twice.  Requisitions  can  sometimes  be  held  up  in  this  process 
if  the  ship  is  entering  a  maintenance  availability  or  shipyard  period  and  material 
deliveries  to  the  ship  are  being  routed  to  another  location. 

Once  a  requisition  clears  the  PMICS  and  NAVSUP  OS  A  screens,  requisitions  are 
split  between  high  and  low  value.  (High  value  is  any  requisition  with  an  extended  value 
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of  more  than  $300.)  During  periods  when  defense  budgets  were  not  so  tight,  low-value 
requisitions  would  then  immediately  be  funded  and,  if  material  were  available,  they 
would  be  sent  to  the  ships.  High-value  requisitions  are  caught,  and  a  re-evaluation  of 
each  allowance  is  conducted  to  ensure  that  the  allowance  is  still  valid  before  funding  is 
applied.  Since  the  process  up  to  this  point  can  take  up  to  45  days,  it  is  possible  that  a  new 
allowance  identified  by  the  CILS-TAT  may  have  become  obsolete  due  to  an  onboard 
system  upgrade  or  a  NIIN  supersession.  The  NSLC  may  also  cancel  a  requisition  during 
this  phase  if  the  re-evaluation  determines  that  the  part  in  question  no  longer  computes  for 
an  allowance  or  that  the  part  has  already  been  provided  by  the  NSLC  or  the  ISEA. 

If  a  requisition  is  cancelled  in  error,  the  ship  is  required  to  contact  the  NSLC  or 
the  item  manager  directly  to  have  the  requisition  reinstated.  At  the  time  of  this  report,  all 
requisitions,  both  high  and  low  value,  generated  as  a  result  of  a  CILS-TAT  are  initially 
given  a  status  code  of  “NM”  by  NSLC,  although  only  high-value  requisitions  are  re¬ 
evaluated.  The  NM  status  code  signifies  that  the  requisitions  are  being  delayed  by  NSLC 
until  funding  becomes  available. 

7.  Prioritization  of  Requisitions 

Once  funding  becomes  available  for  these  new  allowances,  it  is  applied  and 
material  can  start  moving  to  the  ship.  NSLC  does  not  determine  which  requisitions  are 
funded  or  which  CILS-TATs  are  funded  first.  There  may  be  CILS-TAT  requisitions  for 
more  than  one  ship  that  are  awaiting  funding,  and  therefore,  a  detennination  has  to  be 
made  concerning  which  CILS-TATs  are  funded  first.  As  previously  stated,  the 
determination  of  priorities  at  this  level  is  done  by  recommendations  from  the  TYCOM. 
The  TYCOM  typically  sets  its  priorities  based  on  which  ship  is  set  to  deploy  next  and  the 
relative  importance  of  the  mission  that  it  is  set  to  carry  out.  CILS-TAT  requisitions  are 
funded  at  this  point  on  an  all-or-nothing  basis,  meaning  that  if  funds  are  available  for  an 
entire  CILS-TAT  output  for  a  particular  ship,  they  are  released.  Otherwise,  requisitions 
continue  to  hold  with  an  NM  status  until  additional  funds  are  available.  No  prioritization 
exists  for  the  types  (or  criticality)  of  a  particular  part  over  another,  and  as  a  consequence, 
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allowances  that  have  little  impact  on  overall  system  readiness  (as  detennined  by  the 
MCC)  may  be  funded  ahead  of  those  that  could  cripple  a  primary  mission  area. 

H.  NAVY  ENTERPRISE  RESOURCE  PLANNING 

1.  Overview 

In  an  effort  to  improve  the  functionally  and  compatibility  of  such  a  wide 
conglomerate  of  information  systems  across  the  U.S.  Navy  enterprise,  senior  leaders 
endeavored  to  find  an  information  software  package  that  could  help  the  U.S  Navy 
streamline  the  process  of  logistics  information  sharing.  They  were  able  to  work  with 
commercial  vendors  to  develop  a  system  that  mirrors  the  civilian  equivalent  of  ERP.  The 
Navy  ERP  program  uses  a  product  from  SAP  Corporation,  the  largest  provider  of  ERP 
solutions  in  the  world. 

Navy  ERP  is  the  DoN  financial  system  of  record,  meaning  that  it  provides  reliable 
information  for  naval  leadership  to  keep  our  fleet  moving  forward.  Navy  ERP  streamlines 
the  U.S.  Navy’s  business  operations,  namely  financial  and  supply  chain  management 
(U.S.  Navy  ERP,  2013).  In  2010,  Navy  ERP  Release  1.1  (Single  Supply  Solution)  went 
live  on  March  17  at  NAVSUP,  enhancing  the  ability  for  U.S.  Navy  supply  chain 
managers  to  effectively  and  efficiently  provide  Sailors  and  ships  with  the  items  that  they 
need  every  day  (NAVSUP,  2010). 

2.  Impact  on  the  CILS-TAT 

Due  to  the  roll-out  of  Navy  ERP  in  2010,  all  inventory  management  programs 
were  placed  in  a  phasing  plan  to  gradually  implement  the  system  and  mitigate  any 
adverse  effects  that  might  occur  by  rushing  through  the  implementation  process.  The 
highest  priority  was  given  to  the  requirements  that  the  fleet  needed  on  a  day-to-day  basis, 
thereby  relegating  all  re-allowancing  processes  to  a  lower  priority  until  the  system  had 
been  proven.  This  measure  was  taken  to  ensure  that  no  high-priority  requirements  were 
missed  during  the  migration  period.  The  measure  led  to  the  suspension  of  all  CILS- 
TATs  in  2009  until  further  notice. 
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The  Navy  ERP  system  contains  several  improved  capabilities  that  will 
significantly  impact  the  effect  of  the  CILS-TAT  in  the  future.  The  integrated  processes 
within  Navy  ERP  use  a  single  set  of  data,  automatically  disseminate  information  from 
one  entry  to  all  parts  of  the  process  where  it  is  required,  and  make  the  entire  end-to-end 
information  stream  visible  to  managers  with  responsibility  over  the  processes  (U.S.  Navy 
ERP,  2013).  There  are  upgraded  supply  chain  and  financial  management  capabilities  that 
provide  for  better  asset  visibility  across  the  enterprise  as  well  as  better  reporting  of  the 
financial  impacts  of  CILS-TAT  implementation.  The  improved  supply  chain  suite  also 
aids  in  acquiring  the  parts  to  fill  the  newly  added  allowances. 

As  of  2013,  Navy  ERP  has  proven  to  be  a  significant  success  across  the  U.S. 
Navy,  and  senior  leadership  within  the  logistics  community  is  once  again  ready  to  discuss 
the  return  of  the  CILS-TAT.  If  CILS-TAT  is  proven  to  be  a  wise  investment  and  re¬ 
implemented,  it  should  benefit  greatly  from  the  increased  functionality  recognized 
through  the  use  of  Navy  ERP. 
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III.  METHODOLGY 


A.  INTRODUCTION 

Included  in  the  following  chapter  is  a  brief  overview  of  the  methodologies  or 
models  used  in  conducting  the  BCA  regarding  the  effectiveness  of  CILS— TAT.  The 
reasoning  behind  examining  the  following  models  was  threefold.  First  we  utilized  the 
cost-benefit  analysis  (CBA)  to  attach  a  dollar  amount  necessary  to  achieve  a  certain  level 
of  readiness.  Further,  a  CBA  allowed  us  to  determine  the  cost  and  benefits  to  be  weighed 
against  a  perfonnance  metric  defined  by  the  stakeholder.  Next,  we  developed  a 
regression  model  to  evaluate  the  effectiveness  of  CILS— TAT  from  the  before  and  after 
periods.  The  regression  model  was  further  refined  through  the  use  of  cluster  analysis, 
which  enabled  the  grouping  of  data  in  an  attempt  to  find  commonality  experienced  by 
varying  sets  of  ships.  Finally,  we  included  in  this  chapter  an  overview  of  the  knapsack 
model.  This  information  is  contained  within  this  chapter  for  use  in  making  improvements 
to  the  current  CILS— TAT  framework.  The  aforementioned  methods  provide  a 
comprehensive  set  of  tools  for  developing  a  thorough  BCA. 

B.  BUSINESS  CASE  ANALYSIS 

1.  Background 

In  business  and  throughout  the  DoD,  the  use  of  BCA  has  gained  popularity 
through  the  leadership  of  the  Office  of  the  Secretary  of  Defense  (OSD)  to  support 
strategy  decisions  regarding  program  implementation.  A  BCA  provides  a  best-value 
analysis  that  considers  not  only  cost  but  also  other  quantifiable  and  non-quantifiable 
factors  supporting  an  investment  decision  (DAU,  2013).  Depending  on  the  type  of  BCA 
it  may  be  used  throughout  the  life  cycle  of  the  project.  Specifically,  the  BCA  should  be 
used  in  further  decisions  to  sustain  or  enhance  the  solution  and  to  refine  estimation  of 
benefits  and  costs  for  future  projects  in  the  organization  (DAU,  2013).  The  BCA  can  be 
thought  of  as  an  expanded  CBA  with  the  intent  of  detennining  a  best-value  solution.  The 
BCA  process  goes  beyond  cost/benefit  or  traditional  economic  analyses  by  documenting 
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how  each  alternative  fulfills  the  strategic  objectives  of  the  program  and  the  resulting 
impact  on  stakeholders  (DAU,  2013). 

Within  the  BCA,  a  CBA  is  used  as  a  method  to  aid  decision-makers  in 
quantifying  differences  between  projects.  It  provides  a  comparative  assessment  of  all 
benefits  anticipated  and  the  costs  incurred  in  various  iterations.  CBAs  thereby  allow 
decision-makers  to  pick  the  optimal  solution  for  the  allocation  of  scarce  resources. 

When  conducting  a  CBA,  three  frames  of  reference  can  be  used  to  approach  the 
analysis:  ex  ante,  ex  post,  and  in  medias  res  (Boardman,  1996,  p.  3).  These  different 
reference  frames  define  the  time  at  which  the  analysis  is  completed.  In  ex  ante  analysis,  a 
CBA  is  conducted  prior  to  the  start  of  a  project.  The  advantage  of  ex  ante  analysis  is  that 
it  facilitates  the  decision-maker  to  ensure  better  decisions  about  the  allocation  of 
resources;  however,  it  does  not  provide  the  most  complete  understanding  of  the  actual 
benefits  or  cost  assumed — there  is  a  high  degree  of  uncertainty.  An  in  medias  res 
analysis  is  completed  as  the  project  progresses.  While  this  method  provides  a  reduction 
of  uncertainty,  it  reduces  the  ability  of  the  decision-maker  to  have  full  control  of  the 
proper  allocation  of  scarce  resources.  If  a  project  has  been  completed,  an  ex  post  CBA 
can  be  conducted.  This  method  does  not  provide  a  decision-maker  with  the  ability  to 
allocate  resources,  since  they  have  already  been  expensed,  but  is  the  most  accurate 
method  to  understand  the  actual  impacts  of  benefits  and  costs.  For  most  companies,  ex 
post  analysis  is  not  an  option  because  of  the  large  monetary  investment  in  conducting  a 
CBA.  In  general,  most  companies  or  government  agencies  rely  on  ex  ante  analysis  with 
an  understanding  that  there  is  an  added  degree  of  risk  due  to  inherent  uncertainties. 
Looking  further  into  the  CBA  process,  the  framework  for  the  analysis  consists  of  nine 
steps  (Boardman,  1996,  pp.  6-24).  The  following  is  a  breakdown  of  each  step: 

2.  The  Steps  of  a  Cost-Benefit  Analysis 

The  following  steps  describe  a  typical  CBA: 

1.  Decide  whose  benefits  and  cost  count.  In  this  step,  the  analysts  must 
choose  the  scope  of  the  analysis  and  determine  the  target  group.  Specifically,  the 
analysts  must  decide  if  they  are  looking  from  the  perspective  of  a  guardian.  This  decision 
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becomes  integral  in  later  steps  because  it  will  define  how  costs  or  benefits  are  viewed — a 
benefit  to  a  guardian  may  be  a  cost  to  a  spender.  In  general,  guardians  ignore 
nonfinancial  social  benefits.  For  social  benefits  in  the  federal  government,  it  is  assumed 
that  the  analysis  is  being  completed  from  the  spender’s  (society’s)  perspective  vice  the 
guardian’s  (federal  government’s)  perspective  (OMB,  1992). 

2.  Select  the  portfolio  of  alternative  projects.  This  step  allows  the  analysts  to 
bound  the  project  being  analyzed.  For  simplicity,  only  one  project  is  analyzed  at  a 
time — in  theory,  there  are  infinite  numbers  of  alternatives.  This  step  mitigates 
uncertainty  associated  with  complicated  relationships  among  parameters  being 
manipulated.  As  the  number  of  possible  values  increases  among  alternative  projects,  the 
overall  alternatives  increase  exponentially.  As  Boardman  (1996)  observed,  “If  there  were 
n  dimensions,  each  with  k  possible  values,  there  would  be  kn  alternatives  . . .  there  would 
be  27  mutually  exclusive  alternatives.  Neither  decision  makers  nor  analysts  can 
cognitively  handle  comparison  among  such  a  large  number  of  alternatives”  (p.  13). 
Often,  external  constraints  further  restrict  the  project  from  reaching  the  optimal  output 
levels.  The  analysts  must  be  cognizant  of  these  factors  and  what  limitations  are  being 
imposed. 

3.  Catalog  potential  (physical)  impacts,  and  select  measurement  indicators. 
Within  this  step,  the  analysts  define  variables,  both  tangible  and  intangible,  that  are  being 
considered  and  quantify  their  impact  to  the  overall  project.  Chosen  variables  must  ensure 
that  there  is  a  cause-and-effect  relationship  between  a  tangible  outcome  and  a  society. 
When  defining  variables,  analysts  must  explicitly  state  all  assumptions,  especially  if 
assumptions  are  made  about  future  benefits  or  costs  (OMB,  1992).  Once  the  list  is 
collected,  the  variables  are  aggregated  as  either  a  benefit  or  a  cost. 

4.  Predict  quantitative  impacts  over  the  life  of  the  project.  This  step  takes 
each  variable  impact  and  projects  its  value  and  changes  over  the  life  of  the  project.  In 
some  instances,  the  projections  are  taken  past  the  life  of  a  project  if  the  project  is 
anticipated  to  have  a  continued  impact  on  alternate  projects  following  termination.  This 
section  looks  to  correlate  the  impact  of  a  variable  to  a  tangible  value — for  example,  lives 


31 


saved  per  year  or  part  reduction  per  year.  If  existing  projects  are  available,  historical  data 
can  be  used  as  a  baseline  to  extrapolate  the  impact  of  possible  changes. 

5.  Monetize  (attach  a  dollar  value  to)  all  impacts.  The  goal  of  this  step  is  to 
associate  a  monetary  amount  to  all  impacts  in  terms  of  the  dollar  amount  saved  or  lost  for 
each  variable.  This  goal  ensures  a  common  set  of  units  for  comparing  one  variable  to 
another.  The  monetary  amount  associated  with  an  impact  is  often  valued  based  on  a 
willingness  to  pay  (OMB,  1992).  This  relationship  can  be  extrapolated,  for  a  well- 
defined  product,  from  market  prices.  Variables  such  as  parts  reduction  or  labor  reduction 
will  be  monetized  based  on  the  market  price  for  labor  or  the  price  of  each  part  reduced. 

6.  Discount  for  time  to  find  present  values.  If  a  project  is  expected  to  have 
benefits  or  costs  realized  over  the  course  of  the  project’s  life  (years),  these  future  values 
must  be  aggregated  in  their  present  value  (PV)  for  comparison.  The  PV  of  a  future  cost 
or  benefit  can  be  calculated  using  the  PV  equation  (see  Figure  6). 


, ,  future  Value  of  Benefit  or  Cost 

Present  Value  - 

(l+d)r 

Figure  6.  Present  Value  Equation 
(From  Brealey,  Myers,  &  Allen,  201 1,  p.  104) 

In  Figure  6,  d  represents  the  opportunity  cost  of  capital  and  t  represents  the 
number  of  years  in  the  future  that  the  benefit  or  cost  is  realized  (Brealey  et  ah,  2011,  p. 
104).  The  opportunity  cost  of  capital  represents  the  expected  return  not  realized  because 
of  a  project  investment  compared  to  financial  securities  (Brealey  et  ah,  2011,  pp.  G-l  1). 

7.  Sum:  Add  up  the  benefits  and  costs.  Using  the  PV  of  each  variable,  the 
analysts  sum  up  all  benefits  and  costs  to  calculate  the  net  present  value  (NPV)  of  each 
alternative  (see  Figure  7). 


Net  Present  Value 

=  ^  PV  c: 

>t  Benefits  -  ^  PV  of  C 

osts 

Figure  7.  Net  Present  Value  Equation 
(From  Brealey  et  ah,  201 1,  p.  104) 
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8.  Perform  sensitivity  analysis.  A  sensitivity  analysis  allows  the  analysts  to 
capture  the  impact  of  uncertainty  for  each  variable  within  a  project.  This  analysis  will 
identify  to  a  decision-maker  what  variables  can  absorb  higher  degrees  of  uncertainty  with 
marginal  impacts  to  the  outcome.  It  is  very  rare  that  the  impact  of  a  variable  or  its 
valuation  per  unit  impact  is  known  completely.  A  sensitivity  analysis  can  be  completed 
by  either  manually  adjusting  variables  to  see  their  impact  to  the  output  or  by  conducting  a 
model  analysis  (Ragsdale,  2008,  p.  136). 

9.  Recommend  the  alternative  with  the  largest  NPV.  When  choosing 
between  alternatives,  the  decision-maker  should  choose  the  project  or  alternative  with  the 
highest  NPV.  If  the  decision  is  between  all  negative  NPVs,  the  project  or  alternative  with 
the  lowest  negative  value  should  be  chosen.  After  the  analysis,  there  may  be  an  instance 
where  doing  the  status  quo  will  result  in  a  higher  NPV — sometimes  doing  nothing 
different  is  a  better  option. 

C.  KNAPSACK  PROBLEM 

1.  Introduction 

Integer  and  combinatorial  optimization  deals  with  the  problems  of  maximizing  or 
minimizing  a  function  of  many  variables  subject  to  (a)  inequality  and  equality  constraints 
and  (b)  integrality  restrictions  on  some  or  all  of  the  variables.  Because  of  the  robustness 
of  the  general  model,  a  remarkably  rich  variety  of  problems  can  be  represented  by 
discrete  optimization  models  (Nemhauser  &  Wolsey,  1988). 

The  knapsack  problem  (KP)  is  founded  in  the  scope  of  integer  and  combinatorial 
optimization.  Suppose  that  a  hitchhiker  has  to  fill  up  his  knapsack  by  selecting  from 
among  various  possible  objects  those  which  will  give  him  maximum  comfort.  This  very 
rudimentary  question  formulates  the  basis  of  the  KP.  According  to  Martello  and  Toth 
(1990),  the  KP  can  be  mathematically  formulated  by  numbering  the  objects  from  1  to  n 
and  introducing  a  vector  of  binary  variables  Xj  (j  —  1, ...  ,n)  with  the  following 
assignments: 

x  1  if  object  j  is  selected;  _ 

J  to  otherwise. 
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Then,  if  pj  is  a  measure  of  the  comfort  given  by  object  j,Wj,  its  size,  and  c  (the  size  of 
the  knapsack),  our  problem  will  be  to  select,  from  among  all  binary  vectors  x  satisfying 
the  constraint 

SjUwjXj  <  c,  (3) 

the  one  which  maximizes  the  objective  function 

EJLiPjXj.  (4) 

The  KP  has  attracted  and  been  intensively  studied  by  both  theorists  and 
practitioners.  Nemhauser  and  Wolsey  (1988)  espoused  that  an  important  and  widespread 
area  of  application  concerns  the  management  and  efficient  use  of  scarce  resources  to 
increase  productivity.  These  applications  include  operational  problems  such  as  the 
distribution  of  goods,  production  scheduling,  and  machine  sequencing.  They  also  include 
planning  problems  such  as  capital  budgeting,  facility  location,  and  portfolio  analysis. 

For  the  purposes  of  this  research,  we  address  a  capital  budgeting  model  with  the 
problem  of  selecting  among  various  allowancing  possibilities  so  as  to  maximize  the  total 
operational  readiness  without  exceeding  the  available  funds.  According  to  Christolides, 
Mingozzi,  Sandi,  and  Toth  (1979),  this  model  can  be  directly  expressed  as  a  zero-one  (0- 
1)  KP  (each  allowance  possibility  is  either  accepted  or  rejected). 

2.  Zero-One  Knapsack  Problem 

The  0-1  KP  is  the  most  important  KP  and  one  of  the  most  intensively  studied 
discrete  programming  problems.  According  to  Martello  and  Toth  (1990),  the  0-1,  or 
binary,  KP  is  given  a  set  of  n  items  and  the  knapsack,  with 

pj  =  profit  of  item  j, 

wj  —  weight  of  item  j, 

c  —  capacity  of  the  knapsack. 

Select  a  subset  of  the  items  so  as  to 

maximize  Z  —  XjLi  Pjxj  (5) 
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subject  to 


SjLiWjXj  <  c 


(6) 


Xj  —  0  or  1  j  E  N  —  (1, ... ,  n}, 

where 


X; 


1  if  item  j  is  selected; 
0  otherwise. 


The  reason  for  such  an  interest  basically  derives  from  three  facts;  (a)  the  0-1  KP  can  be 
viewed  as  the  simplest  integer  linear  programming  problem,  (b)  it  appears  as  a 
subproblem  in  many  more  complex  problems,  and  (c)  it  may  represent  a  great  many 
practical  situations  (Martello  &  Toth,  1990). 


3.  Bounded  and  Unbounded  Knapsack  Problem 

One  common  variant  of  the  0-1  KP  model  is  that  each  item  can  be  chosen 
multiple  times.  The  bounded  knapsack  problem  (BKP)  specifies  that  for  each  item  j,  an 
upper  bound  bj  (which  may  be  a  positive  integer,  or  infinity)  is  on  the  number  of  times 
that  item  j  can  be  selected  (Martello  &  Toth,  1990).  The  BKP  equation,  as  defined  by 
Martello  and  Toth  (1990),  asserts  to 


maximize 

2  =  Z”=i  PjXj 

(7) 

subject  to 

Z"=l  WjXj  <  c, 

(8) 

0  <  Xj  <  bj  and  integer,)  E  N  —  { 1,  ...,n}. 

The  BKP  is 

a  generalization  of  the  0-1  KP,  in  which  bj  =  1  for  all  j  E  N. 

We  assume, 

without  loss  of  generality,  that 

Pj,  Wj,  bj  and  c  are  positive  integers, 

Ey=i  bj  Wj  >  c, 

(9) 

bjWj  <  c  for  j  E  N. 

(10) 

The  unbounded  knapsack  problem  (UKP),  sometimes  referred  to  as  the  integer 
KP,  does  not  put  any  upper  bounds  on  the  number  of  times  that  an  item  may  be  selected 
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(Christofides  et  al.,  1979).  This  type  of  scenario  might  be  applicable  to  the  allowancing 
product  in  that  the  allowancing  model  determines  that  multiple  assets  of  the  same  line 
item  are  necessary  to  support  the  demand  signal  of  the  ship.  According  to  Martello  and 
Toth  (1990),  the  model  seeks  to 


maximize 

Z  =  £"=1  PjXj 

(11) 

subject  to 

ZjLi  W|Xj  <  c, 

(12) 

Xj  >  0  and  integer,  j  G  n  =  (1, ... ,  n}. 

Through  the  use  of  combinatorial  optimization,  any  possible  revision  of  the 
current  CILS-TAT  model  would  be  streamlined  to  ensure  the  greatest  yield  for  the 
financial  investment.  The  application  of  the  KP  model  to  this  project  is  vitally  necessary 
to  maximize  the  effectiveness  of  the  allowance  products  developed  for  U.S.  Navy  ships. 

D.  REGRESSION  ANALYSIS 

1.  Overview 

Taken  from  a  very  broad  vantage  point,  regression  analysis  can  be  understood  as  a 
statistical  tool  for  the  estimation  of  relationships  between  variables.  It  includes  many 
techniques  for  modeling  and  analyzing  several  variables  when  the  focus  is  on  the  specific 
relationship  between  a  dependent  variable  and  one  or  more  independent  variables.  More 
specifically,  regression  analysis  helps  to  explain  how  the  typical  value  of  the  dependent 
variable  changes  when  any  one  of  the  independent  variables  is  varied  while  the  other 
independent  variables  are  held  fixed  (Lind  &  Mason,  1993).  Regression  analysis  can 
have  varying  uses,  ranging  from  prediction  (to  include  the  forecasting  of  time-series 
data),  inference,  hypothesis  testing,  and  modeling  of  causal  relationships  (Anderson, 
Sweeney,  &  Williams,  2000). 

Linear  regression  was  the  first  type  of  regression  analysis  to  be  studied  rigorously 
and  to  be  used  extensively  in  practical  applications.  According  to  Anderson  et  al.  (2000), 
the  equation  includes  the  effects  or  regression  coefficients  (P),  a  dependent  variable  (y), 
independent  variables  (x),  and  the  error  term  or  noise  (s).  A  regression  model  relates  the 
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dependent  variable  y  to  a  function  of  x  and  P,  written  y  =  f(x,  /?).  The  regression 
analysis  model  can  have  a  single  independent  variable  or  multiple  independent  variables 
as  seen  in  Figure  8. 


y  =  Po+  Pix+  s 

Figure  8.  Linear  Regression  Analysis  Equation 
(From  Lind  &  Mason,  1993) 

With  the  development  of  a  regression  analysis,  the  researcher  must  develop  a  set 
of  assumptions  regarding  the  probability  distribution  of  the  errors  that  must  hold  in  order 
for  the  model  to  be  effective.  Statistical  tests  are  then  made  on  the  basis  of  these 
assumptions  (Lind  &  Mason,  1993). 

It  is  worth  mentioning  here  a  couple  of  key  assumptions  that  must  be  considered 
when  evaluating  the  output  from  the  model.  The  first  assumption  that  must  be  addressed 
is  the  presence  of  multicollinearity,  which  is  defined  as  the  problem  where  changes  in 
two  variables  are  nevertheless  highly  correlated — to  the  point  that  it  is  difficult  to 
separate  their  effects  on  the  dependent  variable  (Sykes,  1993).  The  other  assumption  that 
must  be  addressed  is  the  presence  of  normality.  The  evaluation  of  normality  is  done  by 
calculating  the  random  error  in  the  relationship  between  the  independent  variables  and 
the  dependent  variable  in  a  regression  model;  the  random  error  should  be  normally 
distributed. 

Once  the  regression  model  is  developed,  the  researcher  must  assess  the  statistical 
significance  of  the  estimated  relationship  to  determine  the  strength  of  the  relationship 
predicted  by  the  data  against  the  true  relationship.  Along  with  determining  the  statistical 
significance,  the  investigator  must  detennine  the  goodness  of  fit  for  the  model  by 
evaluating  the  R  value.  The  R  value  is  a  number  between  0  and  1  that  describes  how 
well  a  regression  fits  a  data  set.  Once  the  model  is  developed,  assumptions  have  been 
validated,  and  the  statistical  significance  and  goodness  of  fit  has  been  proven,  the  model 
is  ready  to  be  used  for  various  applications  (Anderson  et  ah,  2000). 
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2.  Linear  Regression 

When  developing  a  simple  linear  regression,  the  researcher  must  determine  which 
variables  of  interest  must  be  taken  from  the  data  set  to  satisfy  the  question  posed  by  the 
hypotheses.  This  data  will  then  be  graphically  depicted  using  a  scatter  plot  diagram.  The 
resulting  display  will  give  the  researcher  the  ability  to  easily  detennine  the  general 
correlation  between  the  two  variables  when  the  other  covariates  are  held  fixed. 

The  regression  is  further  defined  through  the  use  of  the  least  squares  method  to  fit 
a  line  to  the  distribution.  We  call  the  estimate  of  the  line’s  intercept  bo  and  that  of  the 
line’s  slope  bi.  The  estimated  or  predicted  value  of  y  is  denoted  as  y.  Within  the  scatter 
plot,  the  hypothesized  relationship  thus  implies  that  somewhere  on  the  diagram  may  be 
found  a  line  with  the  equation  y  —  b0  +  hxx.  In  the  least  squares  method,  we  minimize 
the  sum  of  squared  differences  between  y  and  y.  Then  we  define  a  residual  for 
observation  i  to  be  et  —  yt  —  yL  and  minimize  the  sum  of  squared  errors,  £f=1  ef 
(Anderson  et  al.,  2000). 

The  same  fundamental  equation  can  also  be  applied  when  multiple  independent 
variables  are  introduced.  The  technique  is  called  multiple  regression,  and  it  allows 
additional  factors  to  enter  the  analysis  separately  so  that  the  effect  of  each  can  be 
estimated.  The  technique  is  valuable  for  quantifying  the  impact  of  various  simultaneous 
influences  upon  a  single  independent  variable  (Sykes,  1993).  Furthermore,  because  of 
omitted  variables  bias  with  simple  regression,  multiple  regression  is  often  essential,  even 
when  the  researcher  is  only  interested  in  the  effects  of  one  of  the  independent  variables 
(Sykes,  1993).  The  general  form  of  the  model  is  y  —  /?0  +  ($1x1  +  /?2x2  +  — I-  /?pxp  + 
e,  and  the  estimated  relationship  is  y  —  b0  +  b1x1  +  b2x2  +  — f  bpxp.  We  use  the  least 
squares  method  to  find  the  values  of  bo,  bi...  bp  that  minimizes  the  sum  of  the  squared 
differences  between  yt  —  yt  (Lind  &  Mason,  1993). 

3.  Assumptions 

As  with  any  model,  there  are  fundamental  assumptions  that  must  be  understood 
before  the  researcher  can  proceed  with  the  interpretation  of  the  results.  There  are  six 

important  assumptions  centered  on  regression  analysis. 
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1 .  Relationship  possesses  linearity. 

2.  Error  terms  (s)  are  normally  distributed. 

3.  At  every  x  value,  the  error  terms  have  constant  variance 
(homoscedasticity). 

4.  The  error  terms  are  independent  of  each  other. 

5.  There  is  a  lack  of  multicollinearity. 

For  the  regression  to  possess  linearity,  the  mean  of  the  response  variable  is  a 
linear  combination  of  the  parameters  (regression  coefficients)  and  the  predictor  variables. 
Because  the  predictor  variables  are  treated  as  fixed  values,  linearity  is  really  only  a 
restriction  on  the  parameters  (Noether,  1971).  In  order  to  check  for  model  linearity  and 
constant  variance,  a  scatter  plot  would  be  used.  This  is  done  by  plotting  the  residuals 
against  the  predicted  values.  The  researcher  would  hope  to  see  a  plot  that  reveals  no 
patterns;  in  other  words,  he  or  she  wants  the  plot  to  have  a  lot  of  randomly  distributed 
points.  If  the  error  terms  have  different  variances,  the  researcher  would  see  the  spread  in 
the  residuals  changing  as  a  function  of  the  predicted  value  (Anderson  et  al.,  2000). 

To  test  the  assumption  that  error  terms  are  normally  distributed,  the  researcher 
must  evaluate  the  residuals.  This  process  is  conducted  by  constructing  a  histogram  of  the 
residuals.  If  the  distribution  looks  bell-shaped,  the  researcher  can  feel  comfortable  that 
the  error  terms  are  close  to  normally  distributed  (Anderson  et  al.,  2000). 

The  final  assumption  in  linear  regression  is  that  the  independent  variables  are 
truly  independent  of  each  other.  The  violation  of  this  assumption  is  referred  to  as 
multicollinearity.  If  the  researcher  is  only  interested  in  prediction,  then  multicollinearity 
may  not  represent  a  large  problem.  If  the  researcher  is  trying  to  explain  the  relationships 
between  dependent  and  independent  variables,  it  does  cause  problems  (Lind  &  Mason, 
1993). 

The  main  problem  is  that  the  standard  error  of  the  regression  coefficients  is  highly 
inflated;  hence,  the  estimated  regression  coefficients  have  large  sampling  variability. 
Estimated  regression  coefficients  tend  to  vary  widely  from  one  sample  to  the  next  when 
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the  independent  variables  are  highly  correlated.  Another  problem  is  the  interpretation  of 
the  estimated  coefficients.  When  the  explanatory  variables  are  correlated,  the  researcher 
cannot  change  one  variable  without  the  correlated  variable(s)  changing  at  the  same  time. 

There  are  two  ways  of  identifying  multicollinearity  within  the  model,  and  the 
approach  varies  depending  on  preference.  The  researcher  can  either  look  for  the  effects 
of  the  correlation  or  focus  on  the  causes.  One  example  of  looking  for  effects  might  be 
identifying  large  changes  in  the  estimated  regression  coefficients  when  a  variable  is 
added  or  deleted.  An  example  of  looking  for  causes  would  be  to  recognize  large 
correlation  coefficients  between  independent  variables  in  the  correlation  matrix  (Neter, 
Wasserman,  &  Kutner,  1990). 

4.  Statistical  Inference  and  Goodness  of  Fit 

The  process  of  determining  the  statistical  inference  of  the  model  is  done  by 
looking  at  the  goodness  of  fit  of  the  model  and  whether  or  not  the  overall  relationship  is 
significant.  In  other  words,  is  the  dependent  variable  related  to  any  of  the  independent 
variables?  Variables  are  evaluated  by  testing  a  hypothesis  of  Ho  and  Ha.  The  hypothesis 
is  tested  by  comparing  the  amount  of  variation  explained  by  the  independent  variables  to 
the  amount  of  variation  left  unexplained.  The  unexplained  variance  is  the  residual  mean 
square.  The  explained  portion  is  referred  to  as  the  regression  mean  square,  and  the  F 
statistic  is  the  ratio  of  explained  to  unexplained  variance  (Anderson  et  ah,  2000). 

All  parameters  were  analyzed  using  the  Welch  Two-Sample  /-test.  This  test 
method  assumes  unequal  variance  between  data  sets  and  that  the  two  data  sets  are  not 
paired.  Under  the  assumption  of  unequal  variances,  the  denominator  of  the  /-test  is  not  a 
function  of  the  pooled  variance,  as  would  be  the  case  in  the  Student’s  /-test  (Keller, 
2009).  Equations  13  and  14  give  the  equation  for  the  Welch  t-statistic. 


Xi-Xi 
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V  = 


/  rr  2  C  2\2 

U,  n2) 


s ; 


NfW- 1)  A22(A2-1) 


(14) 


This  test  was  completed  using  the  /-test  function  in  the  stats  package  of  R  Studio 
software.  Each  data  set  was  evaluated  using  three  different  hypotheses  tests: 


Two  Sided  < 


H0  •  M Before  After  ~  ® 
^  ^  Before  ^ After  ^  ^ 


Less  < 


H0  ‘M  Before  M After  ~  ^ 

-M Before  ~  P After  <  ^ 


Greater  { 


H0  'M. Before  H After  ~  ^ 

Before  ~  M After  >  ^ 


(15) 


(16) 


(17) 


Using  these  three  tests  ensures  that  any  movement,  positive  or  negative,  in  the 
data  set  means  is  fully  captured  in  the  analysis.  In  each  case,  a  95%  confidence  interval 
(Type  I  error  <  0.05)  was  used  as  a  baseline  metric  to  establish  statistically  significant 
changes  in  the  data  sets. 


E.  CLUSTER  ANALYSIS 
1,  Overview 

Cluster  analysis  is  a  subset  of  data  analysis  tools  called  data  mining.  This 
analysis  seeks,  by  grouping  information,  to  find  the  interrelationships  across  various 
parameters  or  variables  to  understand  the  structure  of  data  sets.  Cluster  analysis  places 
data  of  similar  values  together  to  create  a  series  of  A  homogeneous  groupings  (Ye,  2003). 
For  the  purposes  of  this  project,  in  an  effort  to  understand  why  certain  ships  experienced 
a  statistically  significant  change  in  allowance  effectiveness,  cluster  analysis  is  used  to 
find  the  relationship  among  various  source  codes  and  allowance  effectiveness.  For 

example,  for  the  ships  that  achieved  a  statistically  significant  change  in  allowance 
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effectiveness,  was  it  the  respective  change  in  source  code  A  or  source  code  G  which 
caused  the  overall  change  in  allowance  effectiveness?  What  are  the  differentiating  factors 
among  the  ships?  Cluster  analysis  will  show  the  thread  among  parameters  which  causes 
a  ship  to  see  statistically  significant  change  in  allowance  effectiveness. 

2.  K-means  Algorithm 

K-means  falls  under  the  partitional  method  of  cluster  analysis.  In  this  analysis, 
data  is  split  into  user-defined  K  clusters.  The  correct  number  of  clusters  is  a  subjective 
value,  depending  on  constraints  within  the  application  or  oftentimes  found  by  iterating 
through  various  possibilities  to  find  the  best  match  (Ye,  2003).  The  overall  goal  is  to  find 
a  K  value  that  partitions  the  data  into  rational  groupings  with  minimum  overlap  among 
clusters  and  no  empty  clusters. 

The  k-means  algorithm  uses  a  five-step  process,  seen  in  Figure  9,  to  identify  the 
location  and  size  of  the  clusters.  In  the  first  step,  K  number  of  cluster  seed  points  are 
identified  from  the  data  set  and  represent  an  initial  estimate  of  the  location  of  the  center 
of  the  clusters  (initial  centroids). 


Basic  K-means  algorithm 


1 :  Select  K  points  as  initial  centroids. 

2:  repeat 

3: From  K  cluster  by  assigning  each  point  to  its  closest  centroid. 

4: Recompute  the  centroid  of  each  cluster. 

5:  until  Centroids  do  not  change 

Figure  9.  Cluster  Analysis — The  Basic  K-means  Algorithm 

(From  Ye,  2003) 

Once  the  location  of  the  clusters  has  been  identified,  each  of  the  data  points  is 
then  associated  with  a  respective  cluster  based  on  its  proximity  to  the  cluster  centers. 
Fundamentally,  the  proximity  calculation  is  an  optimization  problem  whose  objective 
function  is  to  minimize  the  sum  of  the  squared  error  (SSE),  Equation  18,  based  on  the 
Euclidian  distance  of  each  point  to  the  cluster  center  (Ye,  2003). 
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K 

min  irn  i/e  SSE  =  ^  ^  dist[ci  ,x)  (18) 

/= 1  xsCj 

Turning  to  geometry,  the  Euclidian  distance  represents  nothing  more  than  the  shortest 
distance  between  two  points.  In  n-dimensional  space,  this  distance  is  equal  to  Equation 
19,  where  Ci  and  Xi  represent  the  center  point  of  the  cluster  and  x  represents  a  point  in  the 
data  set: 


n 

dist(c.,x.)=  2>;-x.)2 

V  7=1 


(19) 


Once  the  respective  values  have  been  assigned  to  one  of  the  clusters,  a  new  centroid  is 
calculated  based  on  the  mean  of  the  values  within  the  cluster.  This  process  is  repeated 
until  there  is  no  change  in  the  location  of  the  centroids. 

The  results  of  the  K-means  analysis  are  a  series  K  of  centroid  points  for  each 
cluster  along  with  standard  deviations  of  the  ellipses  representing  a  specified  confidence 
interval.  All  cluster  analyses  in  this  project  were  completed  using  the  k-means  function 
in  R  software  and  the  k-means  cluster  analysis  function  in  JMP  statistical  discovery 
software  from  SAS  using  a  Type  I  error  of  less  than  0.05. 


F.  CONCLUSION 

This  chapter  has  outlined  the  various  analytical  approaches  taken  to  analyze  our 
data  set  in  the  context  of  a  BCA.  In  reviewing  this  chapter,  the  reader  should  have 
confidence  in  the  level  of  analysis  and  research  undertaken.  The  content  should  also 
expand  the  reader’s  knowledge  of  the  techniques  used.  Evaluating  the  impacts  of  CILS— 
TAT  on  individual  ships’  perfonnance  is  a  multivariate  problem  requiring  multiple 
approaches.  The  data  mining  methods  discussed  provide  a  methodical  approach  to 
analyzing  the  numerous  variables  that  have  a  potential  impact  on  ship  performance  and, 
where  possible,  isolate  CILS— TAT  effects.  These  techniques  are  widely  accepted  as 
relevant  approaches  when  conducting  a  BCA,  and  the  authors  felt  that  the  inclusion  of 
each  was  both  appropriate  and  relevant  to  the  research  questions.  Each  of  the  models 
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selected  provides  a  different  angle  from  which  the  data  can  be  evaluated,  and  the  use  of 
multiple  approaches  ensures  accuracy  and  reliability  of  the  BCA. 
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IV.  DATA  ANALYSIS 


A.  DATA  RETRIEVAL 

The  data  used  during  the  course  of  this  project  was  provided  entirely  by  NSLC. 
The  project  required  a  significant  amount  of  historical  data  for  each  of  the  18  ships  we 
analyzed  and  included  a  summary  of  all  CASREPs  submitted  from  2003-2011,  in-port 
and  underway  steaming  hours,  3M  registered  demand  data  from  2003-2011  covering 
more  than  500,000  material  requests,  monthly  percentages  that  ships  were  free  from 
Category  3  and  Category  4  CASREPs,  and  supply  effectiveness  rates. 

NSLC  also  provided  a  detailed  report  for  each  of  the  18  CILS-TATs  conducted 
during  our  period  of  analysis.  These  reports  contained  the  number  of  items 
recommended  to  be  carried  as  allowances,  the  cost  of  each  item  identified,  and  the 
mission  criticality  code. 

As  we  received  data,  we  grouped  and  normalized  it  to  verify  continuity  of 
information  and  determine  whether  gaps  existed  in  the  analysis  window.  Of  the  most 
critical  data  elements,  the  one  with  the  largest  amount  of  missing  data  was  the  CASREP 
summary  information.  The  project  accounts  for  a  total  of  864  months  of  operational  data 
from  the  18  ships  we  surveyed.  From  those  864  months,  there  were  42  months  in  which 
the  CASREP  data  was  either  missing  or  incomplete  and  could  not  be  provided  by  NSLC. 
Since  these  42  months  account  for  exactly  4.86%  of  our  total  sample,  the  amount  of  data 
provided  was  deemed  sufficient  for  further  analysis. 

Due  to  the  large  amount  of  supporting  data  available,  in  our  analysis,  we  were 
able  to  control  for  a  number  of  variables  that  would  otherwise  bring  the  relevancy  of  our 
conclusions  into  question.  One  variable  that  we  were  unable  to  account  for,  however, 
was  the  amount  of  funding  provided  to  the  ships  for  traditional  comprehensive 
allowancing.  Since  funding  provided  in  this  format  is  assumed  to  vary  from  ship  to  ship, 
the  robustness  of  a  ship’s  allowance  product  will  impact  both  its  supply  effectiveness 
rates  and  its  incidence  of  CASREPs. 
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For  the  purposes  of  this  project,  the  ships  and  specific  time  periods  we  analyzed 
are  listed  in  Table  3.  The  ships  represent  both  the  Atlantic  and  Pacific  fleets  and  are  the 
current  members  of  the  U.S.  Navy’s  BMD  fleet. 


Table  3.  List  of  Ships  and  CILS— TAT  Periods  Analyzed 


Name 

UIC 

Date  Range 

MONTEREY  CG61 

21450 

11/1/2003  -  11/30/2007 

SHILOH  CG  67 

21657 

10/1/2004-  10/31/2008 

LAKE  ERIE  CG  70 

21827 

2/1/2005  -  2/28/2009 

VELLA  GULF  CG  72 

21829 

7/1/2003  -  7/31/2007 

PORT  ROYAL  CG73 

21830 

9/1/2003  -  9/30/2007 

JOHN  PAUL  JONES  DDG53 

21313 

8/1/2004-  8/31/2008 

STOUT  DDG55 

21685 

12/1/2004-  12/31/2008 

JOHNS  MCCAIN  DDG  56 

21686 

5/1/2005  -  5/31/2009 

RUSSELL  DDG  59 

21821 

6/1/2003  -  6/30/2007 

PAUL  HAMILTON  DDG  60 

21822 

10/1/2004-  10/31/2008 

RAMAGE  DDG  61 

21823 

7/1/2003  -  7/31/2007 

FITZGERALD  DDG  62 

21824 

6/1/2004  -  6/30/2008 

BENFOLD  DDG  65 

21940 

10/1/2004-  10/31/2008 

THE  SULLIVANS  DDG  68 

21942 

11/1/2003  -  11/30/2007 

MILIUS  DDG  69 

21943 

3/1/2005  -  3/31/2009 

HOPPER  DDG  70 

21944 

11/1/2003  -  11/30/2007 

DECATUR  DDG  73 

21947 

8  1  2003  -  8  31  2007 

HIGGINS  DDG  76 

21950 

5/1/2003  -  5/31/2007 

B.  CILS-TAT  ANALYSIS  WINDOWS 

Throughout  the  analysis,  all  ships  were  analyzed  individually.  The  overall  data 
set  covers  periods  from  December  2003  to  January  2011.  It  is  worth  noting  that  for  the 
remainder  of  this  analysis,  a  period  represents  a  month  of  time  unless  otherwise  specified. 
For  parameters  that  are  sampled  at  frequencies  greater  then  monthly,  the  data  has  been 
averaged  or  summed  as  necessary  to  ensure  consistent  analysis  periods. 
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For  the  various  ships,  CILS— TAT  was  implemented  at  different  times  during  the 
periods  from  June  2003  to  May  2005.  When  analyzing  the  individual  ships’  parameters, 
the  data  sets  are  broken  up  between  the  24  periods  before  the  implementation  of  CILS— 
TAT  and  the  24  periods  after.  In  total,  the  data  set  covers  48  periods.  Table  4  identifies 
the  applicable  date  ranges  for  the  CILS— TAT  data  set  windows.  The  Before  column  in 
Table  4  indicates  the  month  and  year  that  starts  the  24  months  before  CILS— TAT,  and 
similarly,  the  End  column  indicates  the  final  month  and  year  of  the  24  months  following. 
The  date  of  CILS— TAT  implantation  is  indicated  by  the  Entered  column.  When  grouping 
individual  ships  before  and  after  CILS— TAT,  respective  periods  (0-48)  are  combined 
accordingly,  ensuring  consistency  in  period-to-period  analysis. 


Table  4.  CILS— TAT  Analysis  Windows 


Ship  Name 

Before 

Entered 

End 

USS  Monterey 

Nov/2003 

Nov/2005 

Nov/2007 

USS  Shiloh 

Oct/2004 

Oct/2006 

Oct/2008 

USS  Lake  Erie 

Feb/2005 

Feb/2007 

Feb/2009 

USS  Vella  Gulf 

Jul/2003 

Ju 1/2005 

Jul/2007 

USS  Port  Royal 

Sep/2003 

Sep/2005 

Sep/2007 

USS  John  Paul  Jones 

Aug/2004 

Aug/2006 

Aug/2008 

USS  Stout 

Dec/2004 

Dec/2006 

Dec/2008 

USS  John  S  McCain 

May/2005 

May/2007 

May/2009 

USS  Russell 

Jun/2003 

Jun/2005 

Jun/2007 

USS  Paul  Hamilton 

Oct/2004 

Oct/2006 

Oct/2008 

USS  Ramage 

Jul/2003 

Ju 1/2005 

Jul/2007 

USS  Fitzgerald 

Jun/2004 

Jun/2006 

Jun/2008 

USS  Benfold 

Oct/2004 

Oct/2006 

Oct/2008 

USS  The  Sullivans 

Nov/2003 

Nov/2005 

Nov/2007 

USS  Milius 

Mar/2005 

Mar/2007 

Mar/2009 

USS  Hopper 

Nov/2003 

Nov/2005 

Nov/2007 

USS  Decatur 

Aug/2003 

Aug/2005 

Aug/2007 

USS  Higgins 

May/2003 

May/2005 

May/2007 
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C.  MODEL  OUTPUT  AND  ANALYSIS 

1.  Allowance  Effectiveness  Results 

The  analysis  of  CILS— TAT’s  impact  on  allowance  effectiveness  starts  with 
looking  at  the  descriptive  statistics  for  the  before  and  after  data  sets.  Throughout  this 
evaluation,  the  expected  result  is  that  the  mean  of  the  before  data  set  is  less  than  the  mean 
of  the  after  data  set.  This  is  consistent  with  Hypothesis  2  (Equation  16)  noted  in  the 
Methodology  chapter  under  the  Regression  Analysis  section.  Looking  at  Table  5,  which 
outlines  baseline  statistics  for  the  before  and  after  data  sets,  the  entire  before  data  set  has 
an  average  allowance  effectiveness  of  66.03%,  with  values  ranging  from  100%  to 
12.50%.  Between  the  CG  and  DDG  groups,  the  average  allowance  effectiveness  is 
70.08%  and  64.47%,  respectively.  While  these  values  appear  to  be  dissimilar,  they  are 
well  within  one  standard  deviation  of  each  other  and  the  overall  average,  thereby  making 
them  statistically  similar.  For  purposes  of  comparison,  the  data  shared  between  the  two 
classes  of  ships  are  similar  from  an  allowance  effectiveness  perspective. 

For  the  after  data,  the  overall  averages  range  from  25.30%  to  98.40%,  with  an 
average  of  71.12%.  Similarly,  CG  and  DDG  groups  average  73.97%  and  70.02%, 
respectively.  The  absolute  change  in  means  range  from  -17.57%  to  -0.25%.  Overall,  the 
average  change  in  means  for  the  data  sets  was  -5.09.  Categorically,  CGs  experienced  an 
average  change  of  -3.90,  compared  to  an  average  change  of  -5.56  experienced  by  DDGs. 
To  ensure  consistency  in  the  Welch  Two-Sample  /-test,  the  change  in  mean — as 
calculated  in  Table  5 — is  the  difference  between  the  mean  of  the  before  data  set  minus 
the  mean  of  the  after  data  set.  Therefore,  a  negative  number  in  the  Change  in  Mean 
column  from  Table  5  equates  to  an  increase  in  allowance  effectiveness  from  the  before 
period  to  the  after  period. 
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Table  5.  Summary  of  Changes  in  Means  due  to  CILS— TAT 


Summary  Statistics:  Allowance  Effectiveness  [CILS-TAT] 

Before  [2  years] 

Max  Min  Average  StdDev 

Variance 

Max 

Min 

After  ]2  Years] 

Average  StdDev 

Variance 

Change 
in  Mean 

CG 

61 

85.20 

47.10 

67.87 

11.89 

141.48 

97.60 

59.00 

80.23 

9.15 

83.78 

-12.37 

CG 

67 

93.80 

46.30 

69.71 

12.65 

160.11 

91.50 

45.90 

73.79 

10.49 

110.07 

-4.08 

CG 

CG 

70 

88.30 

56.70 

69.34 

7.67 

58.86 

79.50 

41.10 

65.55 

11.03 

121.75 

3.80 

CG 

72 

86.20 

59.20 

73.80 

7.12 

50.65 

91.70 

25.30 

75.88 

13.79 

190.13 

-2.08 

CG 

73 

83.80 

46.80 

69.67 

9.88 

97.56 

93.60 

48.60 

74.42 

10.96 

120.22 

-4.75 

CG  Overall  |Avg] 

93.80 

46.30 

70.08 

10.11 

102.27 

97.60 

25.30 

73.97 

12.00 

143.99 

-3.90 

DDG 

53 

83.60 

16.90 

56.91 

15.42 

237.75 

90.70 

44.70 

71.01 

15.11 

228.29 

-14.10 

DDG 

55 

100.00 

39.70 

68.09 

14.50 

210.29 

93.40 

46.60 

66.32 

12.57 

157.94 

1.78 

DDG 

56 

86.20 

12.50 

63.70 

19.38 

375.76 

89.60 

41.10 

71.64 

12.48 

155.71 

-7.93 

DDG 

59 

86.00 

37.80 

62.39 

13.71 

188.05 

85.10 

53.30 

71.87 

10.31 

106.24 

-9.48 

DDG 

60 

79.60 

40.80 

62.45 

11.02 

121.43 

89.00 

28.70 

67.10 

15.66 

245.36 

-4.65 

DDG 

61 

86.40 

43.90 

64.71 

10.27 

105.50 

95.70 

29.00 

67.52 

14.50 

210.18 

-2.81 

DDG 

DDG 

62 

98.70 

38.70 

63.28 

17.58 

309.22 

92.50 

46.00 

67.13 

12.95 

167.82 

-3.86 

DDG 

65 

89.10 

15.80 

63.65 

15.23 

231.86 

92.40 

38.40 

73.69 

11.45 

131.01 

-10.04 

DDG 

68 

93.10 

47.20 

67.27 

11.40 

130.07 

82.70 

48.40 

64.30 

10.00 

100.01 

2.97 

DDG 

69 

91.50 

31.60 

66.68 

12.93 

167.24 

98.40 

69.50 

84.26 

7.96 

63.38 

-17.57 

DDG 

70 

98.90 

50.50 

72.57 

15.18 

230.42 

91.20 

43.40 

72.83 

13.17 

173.53 

-0.25 

DDG 

73 

89.30 

43.30 

67.57 

10.43 

108.88 

92.30 

45.10 

69.10 

11.99 

143.76 

-1.53 

DDG 

76 

78.40 

31.70 

58.79 

11.79 

139.07 

92.00 

29.00 

63.53 

16.04 

257.41 

-4.73 

DDG  Overall  [Avg] 

100.00 

12.50 

64.47 

14.30 

204.55 

98.40 

28.70 

70.02 

13.60 

184.83 

-5.56 

All  Ships  Overall  [Avg] 

100.00 

12.50 

66.03 

13.50 

182.17 

98.40 

25.30 

71.12 

13.28 

176.26 

-5.09 

While  all  18  ships  experienced  a  change  in  the  means  of  their  data  sets,  only  six 
ships,  highlighted  in  yellow,  had  a  change  based  on  the  Welch  Two-Sample  Z-test,  which 
was  statistically  significant.  For  the  remaining  ships  that  did  not  meet  the  threshold  for  a 
Type  I  error  of  less  than  0.05,  there  was  not  enough  evidence  to  prove  that  the  variance  in 
the  data  sets  and  the  respective  changes  in  means  was  not  purely  based  on  chance.  The 
intent  of  this  analysis  is  to  show  that  for  six  of  the  ships  in  the  sample,  CILS— TAT  was 
directly  responsible  for  the  change  in  effectiveness.  The  results  of  the  Welch  Two- 
Sample  /-test  are  summarized  in  Table  6. 
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Table  6.  Summary  of  Allowance  Effectiveness  and  the  Welch  Two-Sided  t-Test 


T-Te<t:  Allow 

ance  Effects 

Mean  of 
Before 

Mean  of 
After 

Less  Alternative  -  11  o  :^,rHw  =  fl  J|W  II, :  <  //  (lhr 

Change  In  T  Suthtic  p  Value  DOF  LCL 

Mean 

UCL 

CG  61 

67.87 

80.23 

-12.37 

-4.04 

0.000109 

43.17 

-  OC 

-7.22 

CG  67 

69.71 

73.79 

-4.08 

■122 

0.115242 

44.47 

-  OC 

1.56 

CG  70 

69.34 

65.55 

3.80 

1.38 

0.913032 

41.03 

—  ac 

8.41 

CG  72 

73.80 

75.88 

-2.08 

-0.66 

0.258376 

34.44 

-  ac 

3.28 

CG  73 

69.67 

74.42 

-4.75 

-1.58 

0.060877 

45.51 

-  ac 

0.31 

DIXi  53 

56.91 

71.01 

-14.10 

-3.20 

0.001247 

45.98 

-  ac 

-6.70 

DDG55 

68.09 

66.32 

1.78 

0.45 

0.673693 

45.09 

-  ac 

8.35 

DUG  56 

63.70 

71.64 

-7.93 

-1.69 

0.049876 

2921 

-  ac 

-0.01 

Individual 

DIXi  59 

62.39 

71.87 

-9.48 

-2.71 

0.004861 

42.70 

-  ac 

-3.59 

DDG60 

62.45 

67.10 

-4.65 

-1.19 

0.120735 

41.29 

—  ac 

1.93 

DDG61 

64.71 

67.52 

-2.81 

-0.78 

0.221230 

41.44 

-  ac 

3.29 

DDG  62 

63.28 

67.13 

-3.86 

-0.87 

0.195844 

42.28 

-  ac 

3.64 

DDG  65 

63.65 

73.69 

-10.04 

-2.58 

0.006679 

42.70 

—  ac 

-3.50 

DDG  68 

(,121 

64.30 

2.97 

0.96 

0.828462 

45.23 

—  ac 

8.17 

DDG  69 

66.68 

84.26 

-17.57 

-5.67 

0.000001 

38.24 

-  ac 

-12.35 

DDG  70 

72.57 

72.83 

-0.25 

-0.06 

0.475437 

45.11 

-  ac 

6.64 

DDG  73 

67.57 

69.10 

-1.53 

-0.47 

0.319392 

45.14 

-  ac 

3.92 

DDG  76 

58.79 

63.53 

-4.73 

-1.16 

0.125367 

42.24 

-  ac 

2.10 

The  primary  driver  that  differentiated  statistically  significant  ships  from  those  that 
were  not  statistically  significant  was  a  large  change  in  the  means  of  the  data  sets  coupled 
with  a  decrease  in  the  variance  and  standard  deviation  of  the  data  from  the  before  period 
to  the  after  period.  Figure  10,  in  the  top  graph,  shows  a  boxplot  for  each  ship’s  allowance 
effectiveness  before  (blue)  and  after  (red)  CILS— TAT.  For  each  ship,  the  dot  in  the 
middle  is  the  mean  effectiveness,  and  the  block  represents  the  range  of  +/-  one  standard 
deviation  from  the  mean.  Finally,  the  whiskers  show  the  difference  between  the 
maximum  and  minimum  allowance  effectiveness.  The  blue  line  between  the  two  data 
sets  represents  the  change,  direction  and  magnitude,  of  the  before  and  after  data. 
Similarly,  plotted  in  the  bottom  section  of  Figure  10  are  the  standard  deviations  before 
and  after  for  each  ship.  This  graph  both  identifies  the  change  in  standard  deviation  for  an 
individual  ship  and  shows  these  standard  deviations  in  the  context  of  the  overall  sample 
population. 
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Variance  in  Allowance  Effectiveness  Before  to  After  CILS-TAT 

Before  Cl  LS-TAT  -  After  CILS-TAT 


CG  CG  CG  CG  DOG  DDG  DDG  DDG  DDG  DDG  DDG  DDG  DDG  DOG  DDG  DDG  DOG 

67  70  72  73  S3  55  56  59  60  61  62  65  68  69  70  73  76 

Figure  10.  Variance  in  Allowance  Effectiveness  Before  and  After  CILS-TAT 

For  five  out  of  the  six  ships,  this  is  the  exact  relationship  observed:  the  allowance 
effectiveness  increased  in  all  ships  where  the  change  was  statistically  significant. 
However,  for  DDG  53,  the  standard  deviation  of  the  data  sets  only  changed  marginally. 
For  this  ship,  it  was  the  -14.10%  change  in  allowance  effectiveness  that  was  well  above 

the  upper  control  limit  (UCL)  of  -6.70  that  allowed  this  value  to  be  statistically 

significant.  The  key  is  that  the  ships  that  experienced  a  statistically  significant  change 
saw  less  variance  in  allowance  effectiveness  following  the  implementation  of  CILS- 
TAT.  The  reduction  in  variance  from  before  to  after  indicates  that  CILS-TAT  enabled 
the  ships  to  more  tightly  and  accurately  control  allowance  effectiveness.  Similarly, 
increased  control  in  variance  resulted  in  a  significant  increase  in  the  allowance 
effectiveness  values  post  CILS-TAT — the  ships  were  able  to  maintain  a  higher  value  of 
allowance  effectiveness.  From  a  manager’s  perspective,  we  can  conclude  that  because  of 
CILS-TAT  implementation,  we  noticed  an  improved  level  of  allowance  effectiveness  as 
well  as  a  more  consistent  percentage  from  before  to  after. 
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Given  the  final  results  of  the  before  and  after  data,  CILS— TAT  had  a  direct  and 
positive  impact  on  allowance  effectiveness  for  six  out  of  18  ships.  Of  the  12  remaining 
ships,  nine  saw  a  positive  change  in  allowance  effectiveness,  although  this  change  is  not 
statistically  significant  and  thus  cannot  be  attributed  to  CILS— TAT. 

2.  CASREP  Frequency  and  POTF  Results 

The  CASREP  data  used  represents  the  daily  count  for  Categories  2,  3,  and  4.  For 
review,  CASREPs  reflect  broken  or  inoperable  equipment  onboard  U.S.  Navy  ships.  In 
order  to  aggregate  the  data,  the  CASREPs  were  totaled  over  the  monthly  period  to  be 
consistent  with  the  periodicity  of  allowance  effectiveness  data.  However,  for  the 
purposes  of  CASREP  frequency  analysis,  the  various  categories  of  CASREPs  were 
summed  together  with  C3  and  C4  CASREPs  representing  less  than  20%  of  all  CASREPs. 

Looking  at  Table  7,  seven  out  of  the  18  ships  resulted  in  a  positive  change  in 
mean  (before  minus  after).  Of  these  seven  ships,  only  two  of  the  ships  experienced  a 
change  with  a  Type  I  error  less  than  0.05.  None  of  the  ships  that  experienced  a 
statistically  significant  change  in  allowance  effectiveness  saw  a  similar  result  for 
CASREPs.  Overall,  the  average  change  in  total  CASREPs  was  -0.5. 
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Table  7.  /-Test  Results  for  Total  CASREPs 


T-Test:  Tool  CASREP  |CILS-TAT| 

Hull  Number 

Mean  of 
Before 

Mean  of 

After 

Change  In 
Mean 

T  Statistic 

DOF 

P  Value 

LCL 

UCL 

CG  61 

10.52 

8.43 

2.09 

0.98 

40.S4 

0.166151 

-1.49 

+  00 

DDG  53 

6.24 

7.82 

-1.58 

-0.82 

28.33 

0.790945 

-4.86 

+  OD 

Allowance  DDG  56 

Effectiveness 

3.83 

6.95 

-3.12 

-1.93 

25.83 

0.967970 

-5.87 

+  00 

11.41 

5.00 

6.41 

1.69 

17.27 

0.054497 

-0.18 

+  OD 

DDG  65 

6.00 

5.95 

0.05 

0.03 

39.64 

0.488102 

-2.76 

•f  oo 

DDG  69 

3.89 

4.20 

-0.3! 

-0.25 

34.78 

0.597668 

-2.42 

■f  00 

P  Value  <  0.05  [Avg] 

6.98 

6.39 

0.59 

-0.05 

31.06 

0.510889 

-2.93 

■f  00 

CG  67 

6.58 

7.30 

-0.73 

-0.37 

39.24 

0.643317 

-4.03 

+  00 

CG  70 

5.68 

8.17 

-2.49 

-1.26 

36.14 

0.891559 

-5.83 

+  oo 

CG  72 

9.68 

11.13 

-1.44 

-0.53 

42.64 

0.700913 

-6.01 

+  00 

CG  73 

7.95 

12.21 

-4.26 

-1.61 

39.14 

0.941882 

-8.72 

+  00 

DDG  55 

10.85 

9.32 

1.53 

0.61 

32.28 

0.272666 

-2.71 

+  00 

Allowance  nnr 

Effectiveness 

8.95 

14.96 

-6.01 

-1.71 

30.20 

0.951378 

-11.96 

+  00 

P  Value  >  0.05  DDG  61 

8.35 

6.32 

2.03 

0.88 

40.09 

0.191224 

-1.84 

+  00 

DDG  62 

4.68 

6.61 

-1.92 

-1.78 

39.97 

0.958962 

-3.74 

+  00 

DDG  68 

8.05 

11.57 

-3.52 

-1.86 

38.72 

0.964697 

-6.70 

+  oo 

DDG  70 

3.76 

6.79 

-3.03 

-1.32 

29.72 

0.901880 

-6.91 

+  oo 

DDG  73 

4.65 

3.36 

1.29 

1.55 

39.87 

0.065117 

-0.12 

•f  oo 

DDG  76 

9.25 

4.36 

4.89 

2.23 

30.90 

0.016559 

1.17 

4-  oo 

P  Value  >  0.05  [Avg] 

7.37 

8.51 

-1.14 

-0.43 

36.58 

0.625013 

-4.78 

1  S. 

1  AB  Ships  [Avg] _ 

7.24 

7.80 

-0.S6 

-0.30 

34.74 

0.586972 

-4.17 

+  00 

Turning  to  the  POTF  metric,  which  is  calculated  as  a  quarterly  percentage, 
represents  the  amount  of  time  a  ship  operates  without  a  Category  3  or  4  CASREP.  Of  the 
18  ships  evaluated,  seven  had  a  negative  change  in  POTF.  Using  Table  8,  the  mean  of 
the  After  data  is  greater  than  the  mean  of  the  before  data  with  a  maximum  change  of  - 
29.71.  While  these  ships  experienced  a  change  in  POTF,  only  three  of  the  ships  were 
determined  to  be  statistically  significant.  It  is  important  to  note,  however,  that  POTF  is 
reported  on  a  quarterly  basis,  and  for  the  48  months  analyzed,  there  are  only  16  data 
points,  which  is  considered  a  small  sample  for  analysis. 
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Table  8.  /-Test  Results  for  POTF  From  CASREP 


T-Test:  Percentage  of  Time  Free  from  CASREP  (POTF)  |CILS-TAT] 


Hull 

Before 

Means  of 
After 

Change  in 

T  Statistic 

DOF 

P  Value 

I. Cl. 

UCL 

CG  61 

77.60 

88.63 

-11.03 

-130 

10.04 

0.128694 

—  00 

5.61 

DDG  53 

63.63 

53.00 

10.63 

0.65 

13.03 

0.737322 

—  00 

39.45 

Allowance  DDq  56 

Effectiveness  _  .  _ 

70.00 

72.50 

-2.50 

-0.16 

10.73 

0.436541 

—  ao 

25.00 

P  Value  <0  01  DDG59 

100.00 

40.13 

59.88 

5.49 

7.00 

0.999542 

—  ao 

80.53 

DDG65 

87.29 

54.00 

33.29 

2.97 

9.02 

0.992218 

—  ao 

53.80 

DDG  69 

74.25 

64.50 

9.75 

0.67 

13.8] 

0.744204 

—  ao 

35.26 

P  Value  <  0.05  [Avg] 

78.79 

62.13 

16.67 

1.40 

10.61 

0.673087 

—  00 

39.94 

CG  67 

42.29 

72.00 

-29.71 

-1.88 

10.78 

0.043357 

—  ao 

-1.35 

CG  70 

78.25 

37.00 

41.25 

2.44 

13.99 

0.985796 

—  ao 

70.99 

CG  72 

96.67 

72.75 

23.92 

234 

835 

0.976894 

-  ao 

42.83 

CG  73 

71.00 

48.50 

22.50 

0.99 

7.60 

0.823758 

—  ao 

65.04 

DDG  55 

9.25 

41.63 

-3238 

-2.02 

12.45 

0.032608 

—  ao 

-3.92 

Allowance  nnn  wi 

Effectiveness 

96.14 

73.75 

2239 

3.22 

8.84 

0.994615 

—  ao 

35.17 

P  Value  >0.05  61 

72.00 

80.63 

-8.63 

-0.53 

6.12 

0306526 

—  ao 

22.73 

DDG  62 

67.71 

60.63 

7.09 

0.43 

12.12 

0.663242 

-  ao 

36.33 

DDG  68 

80.80 

70.75 

10.05 

0.58 

10.79 

0.713115 

“  ao 

41.22 

DDG  70 

89.60 

74.63 

14.98 

1.50 

10.60 

0.919007 

—  ao 

32.93 

DDG  73 

68.50 

82.13 

-13.63 

-0.66 

3.94 

0.273569 

—  ao 

30.72 

DDG  76 

59.33 

7838 

-19.04 

-1.93 

8.88 

0.042774 

—  ao 

-0.97 

P  Value  <  0.05  (Avg| 

69.30 

66.06 

3.23 

037 

9.54 

0.564605 

—  ao 

30.98 

All  Ships  (AvgJ 

72.46 

64.7S 

7.71 

0.72 

9.9(1 

0.600766 

-  ao 

33.97 

Overall,  based  on  the  minimal  changes  in  CASREP  frequency  and  the  lack  of  a 
statistically  significant  change  of  POTF,  CILS-TAT  appears  to  have  had  a  negligible 
impact  on  the  number  of  CASREPs  experienced  and  the  amount  of  time  CASREPs  are 
open.  Additionally,  ships  that  saw  an  increase  in  allowance  effectiveness  as  a  result  of 
CILS-TAT  did  not  see  a  reduction  in  CASREP  frequency  or  marked  improvement  in 
POTF.  The  inclusive  statistical  relationship  could  be  explained  by  an  assumption  that  the 
processing  of  CASREPs  onboard  ships  is  often  riddled  with  variable  subjectivity 
introduced  by  the  controllers  of  the  process. 
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3.  Impact  of  CILS— TAT  Spending 


To  measure  the  impact  of  the  dollars  spent  on  CILS— TAT,  and  the  respective 
changes  in  allowance  effectiveness,  we  turn  to  the  amount  of  money  obligated.  Table  9 
shows  a  breakdown  of  the  actual  dollar  amounts  obligated  in  the  year  that  CILS— TAT 
was  implemented.  These  values  have  been  adjusted  to  constant  fiscal  year  (FY)  2003 
(base  year)  dollars  using  the  OPN  inflation  category  found  in  the  2012  Joint  Inflation 
Calculator.  FY2003  was  chosen  as  the  base  year  in  conjunction  with  the  beginning  of  the 
first  CILS— TAT  analysis  periods. 


Table  9.  CILS— TAT  Obligation  Amounts 


Hull 

Number 

Year 

Obligated 

Inflation 

Factor 

Actual 

Obligated  Value 

Actual  Obligated 
Value  (SFY2003) 

CG  61 

2005 

0.9537 

$234,668.92 

$223,801.14 

CG  67 

2006 

0.9250 

$161,646.56 

$149,525.24 

CG  70 

2007 

0.9007 

$184,329.58 

$166,024.67 

CG  72 

2005 

0.9537 

$229,774.69 

$219,133.56 

CG  73 

2005 

0.9537 

$292,576.74 

$279,027.18 

DDG  53 

2006 

0.9250 

$837.01 

$774.25 

DDG  55 

2006 

0.9250 

$751.57 

$695.21 

DDG  56 

2007 

0.9007 

$92,066.71 

$82,924.00 

DDG  59 

2005 

0.9537 

$201,782.00 

$192,437.25 

DDG  60 

2006 

0.9250 

$119,360.84 

$110,410.38 

DDG  61 

2005 

0.9537 

$182,834.45 

$174,367.18 

DDG  62 

2006 

0.9250 

$318,133.93 

$294,278.16 

DDG  65 

2006 

0.9250 

$227,216.30 

$210,178.13 

DDG  68 

2005 

0.9537 

$62,657.90 

$59,756.14 

DDG  69 

2007 

0.9007 

$86,102.12 

$77,551.72 

DDG  70 

2005 

0.9537 

$159,333.84 

$151,954.91 

DDG  73 

2005 

0.9537 

$148,749.17 

$141,860.43 

DDG  76 

2005 

0.9537 

$229,568.54 

$218,936.96 

1 

Total 

$2,932,390.87 

$2,753,636.52 

For  purposes  of  comparison,  Figure  1 1  brings  together  the  allowance 
effectiveness  /-test  results  and  the  dollar  amounts  obligated  after  CILS— TAT.  FY2003 
obligated  values  were  plotted  against  the  observed  change  in  allowance  effectiveness 
means  and  the  respective  /-test  p-value.  Ships  that  were  identified  as  statistically 
significant  are  annotated  by  red  circles. 
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Figure  1 1 .  Explaining  the  Relationship  Between  Amounts  Obligated  and 
the  Significance  of  the  Change  in  Allowance  Effectiveness  for  Each  Ship 


The  top  graph  in  Figure  1 1  shows  how  the  three  variables  interact.  The  bottom 
charts  are  two-dimensional  projections  of  the  various  variables  from  the  top  chart. 
Starting  with  the  amount  obligated  versus  the  change  in  means,  while  a  strong 
relationship  between  these  two  does  not  appear  to  exist,  using  cluster  analysis  and  the  k- 
means  function  in  R  Statistics  software,  the  ships  were  grouped  into  three  clusters  with 
similar  characteristics.  Fifteen  parameters  were  used  as  variables  in  the  cluster  analysis, 
including  obligated  amount,  p-values,  and  changes  in  means  for  the  various  source  codes. 
These  variables  were  used  in  an  effort  to  find  those  parameters  that  differentiate  the  ships 
that  experienced  a  statistically  significant  difference  in  means  and  those  that  did  not.  A 
grouping  of  three  clusters  was  used  because  it  naturally  split  the  /-test  p-values  into  three 
levels  of  Type  I  errors,  with  one  group  centered  at  approximately  0.05. 
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Based  on  this  analysis,  Cluster  1  (blue  ellipse)  encompasses  all  ships  with  a  Type 
I  error  approximately  less  than  0.05.  Cluster  1  is  centered  on  a  p-value  of  0.03,  a  change 
in  means  of  -10.1 1%,  and  an  obligated  amount  of  $147,138.  Within  this  cluster,  the  ships 
can  be  split  again  into  two  groups  based  on  the  amount  that  was  obligated.  In  the  first  set 
(bottom  left),  DDG  53  and  DDG  69  achieved  a  significant  change  in  means  with  an 
obligated  amount  almost  half  of  the  other  statistically  significant  ships — less  than 
$100,000.  Included  in  this  grouping,  DDG  56  did  not  see  as  large  of  a  change  in  means 
for  a  similar  amount  obligated,  but  the  change  was  enough  to  be  statistically  significant. 

In  the  second  grouping  in  the  first  cluster  (top  left)  are  CG  61,  DDG  59,  and  DDG 
65,  with  amounts  obligated  ranging  from  $174,000  to  $210,000.  They  achieved  a  smaller 
change  in  the  means  of  allowance  effectiveness,  with  an  average  change  of  -10.63%.  The 
other  two  clusters  of  ships,  Cluster  2  (red  ellipse)  and  Cluster  3  (green  ellipse), 
encompass  the  remaining  ships  whose  change  in  allowance  effectiveness  was  not  below 
the  p-value  threshold  of  less  than  0.5.  The  important  takeaway  from  this  analysis  is  that 
the  amount  of  money  obligated  appears  to  be  independent  of  a  change  in  allowance 
effectiveness.  This  is  true  in  terms  of  both  magnitude  of  the  change  and  its  statistical 
significance.  For  roughly  the  same  amount  obligated,  Clusters  1  (blue  ellipse)  and  2  (red 
ellipse)  achieved  vastly  different  changes  in  means  and  statistical  significance. 

Cluster  analysis  was  also  used  to  examine  individual  source  codes.  There  are  two 
for  Cluster  1,  source  codes  A  and  C,  for  which  almost  all  ships  experienced  a  significant 
change.  Figure  12  shows  the  relative  magnitude  of  the  change  in  means  and  their 
statistical  significance  for  the  three  clusters.  Note  that  the  values  in  Figure  12  have  been 
nonnalized  to  allow  for  better  comparison  between  variables  with  dissimilar  scales.  It  is 
important  to  note  that  in  terms  of  source  codes  (see  Figure  4)  A,  C,  D,  and  F,  the 
expected  change  in  means  is  negative,  and  for  source  codes  G  and  J,  the  expected  change 
is  positive.  This  indicates  that  for  the  first  four  source  codes  of  A,  C,  D,  and  F,  the  lower 
the  value  for  change  in  means,  the  better,  and  for  the  final  two  source  codes  of  G  and  J, 
the  larger  the  value,  the  better.  For  the  ships  that  are  statistically  significant,  Cluster  1,  it 
is  clear  that  these  ships  experienced  the  largest  changes  in  four  out  of  six  source  codes; 
however,  only  source  codes  A  and  C  experienced  Type  I  errors  less  than  0.05. 
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Source  Code  Description 


Standardized  Allowance  Effectiveness  and  Variable  Clusters  Means 


Cluster  Location  &  95%  Confidence  Ellipse 
Cluster  Number 


P  Value:  Allowance  Effectiveness 
Change  in  Means:  Allowance  Effectiveness 


Source  Code  A 

Source  Code  C 

„ . , ,  Source  Code  D 
P  Value 

Source  Code  F 
Source  Code  G 
Source  Code  J 


0357258 

0.128745 

0.820998 

0.543827 

0.063805 

0.026774 


0.333200 

0.177577 

0.268588 

0.362566 

0.081574 

0.066252 


0.024020 

0.468553 

0.358858 

0.022968 

0.936188 

0.769961 


0.689032 

0.956611 

0.842876 

0.565930 

0.561863 

0.036699 


0.014703 

0.455850 

0.334407 

0.009545 

0.063764 

0.216851 


0.278194 

0.076607 

0.209272 

0.402406 

0.397449 

0.087970 


0.009361 

10.26 

18.09 

5.99 

-15.83 

0.05 

-5.80 

-39.96 


0.139951 

0.378823 

5.99 

1.58 

34.64 

-16.49 

3.09 

-5.52 

4.55 

-13.09 

4.92 

-0.39 

5.38 

1.07 

13.83 

-28.17 

0.014639 

3.05 

31.28 

3.63 

12.53 

3.53 
4.32 
18.19 


0.111190 

0.251522 

3.39 

2.79 

4.13 

29.62 

5.47 

4.10 

9.92 

12.39 

1.41 

2.61 

3.34 

4.31 

11.78 

12.01 

Allowance  List  Material  issues 
from  storeroom  stock 


Non-Allowance  List  Material 
(SIM)  issues  from  storeroom  stock 


Not  Carried  (NC)  repair  parts 
which  are  listed  on  an  APL.  but 
does  not  compute  for  an  allowance 


Allowance  List  Material  Not  In 
Stock  (NISI  when  requested 

Non-Allowance  List  Material 
(SIM)  Not  In  Stock  (NIS)  when 
requested 

Not  Carried  (NC)  repair  parts 
which  are  not  listed  on  an  APL  in 
the  ship's  COS  file 


Change  in 
Means 


Source  Code  A 
Source  Code  C 
Source  Code  D 
Source  Code  F 
Source  Code  G 
Source  Code  J 


Figure  12.  Source  Codes  That  Drive  Allowance  Effectiveness  to  be 
Statistically  Significant  Using  Cluster  Analysis 


Looking  at  Table  10,  which  shows  the  results  of  t-test  analysis  for  the  various 
source  codes  in  conjunction  with  Figure  13  in  terms  of  its  change,  source  code  C  had  the 
lowest  p-value  both  individually  and  when  averaged  across  the  six  statistically  significant 
ships.  However,  in  terms  of  numerical  magnitude  and  the  parameters’  relative  ability  to 
change  the  allowance  effectiveness  equation,  the  primary  parameter  of  concern  is  source 
code  A.  This  can  be  seen  in  the  ellipse  for  change  in  source  code  A,  which  is  centered 
on  -57.73,  while  the  ellipse  for  source  code  C  is  centered  on  -12.18.  In  summary,  it  was 
found  that  source  code  A  had  a  greater  ability  to  impact  the  change  in  allowance 
effectiveness  than  any  other  source  code 

In  keeping  with  this  theme,  overall  changes  in  source  code  G  are  not  statistically 
significant  within  a  95%  confidence  interval.  This  can  be  explained  by  looking  at  the  t- 
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test  results  for  source  code  G.  Looking  at  the  p-values  for  the  greater  than  and  less  than 
hypothesis,  there  does  not  appear  to  be  a  clear  direction  among  all  of  the  six  ships  with 
statistically  significant  change  nor  among  all  ships  for  the  change  in  source  code  G;  out 
of  all  of  the  ships,  33%  were  statistically  significant  with  a  greater  than  hypothesis  and 
17%  with  a  less  than  hypothesis.  In  terms  of  source  code  J,  the  ships  in  Cluster  1  fell 
between  the  other  clusters.  Similarly,  this  is  explained  by  the  ambiguous  direction  of  the 
change  in  means  among  all  ships;  for  the  greater  than  hypothesis,  17%  of  the  ships 
experienced  a  Type  I  error  less  than  0.05,  and  likewise,  under  the  less  than  hypothesis, 
17%  of  the  ships  experienced  a  Type  I  error  less  than  0.05.  Essentially,  the  results  of  the 
G  and  J  source  codes  are  not  consistent  and  not  statistically  significant.  The  data  is  split 
unevenly  in  directionality.  Early  on,  CILS— TAT  affected  A  and  C  but  did  not  affect  the 
G  and  J  source  codes.  According  to  the  framework  of  source  codes,  you  would  expect 
some  correlation  between  A/C  and  G/J.  As  stated  previously,  the  data  analysis  does  not 
reveal  any  correlation  between  the  four  source  codes  of  A,  C,  G,  and  J. 

These  changes  are  consistent  with  the  expected  results  of  CILS— TAT 
implementation  since  the  technique  is  designed  to  increase  the  frequency  of  source  codes 
A  and  C.  If  demands  received  in  the  after  period  are  identical  to  demands  received  in  the 
before  period,  then  one  would  expect  the  occurrence  of  source  codes  A  and  C  to  increase 
while  G  and  J  decreased  in  a  perfectly  inverse  relationship.  In  reality,  however,  there  are 
other  factors  that  influence  the  frequency  of  G  and  J  that  cannot  be  completely  mitigated 
by  CILS— TAT  (e.g.,  poor  configuration  management  practices).  This  proves  that, 
independent  of  the  amount  of  money  obligated,  in  those  ships  that  achieved  a  statistically 
significant  change  in  allowance  effectiveness,  CILS— TAT  was  able  to  accurately  target 
source  codes  A  and  C.  This  implies  that  the  true  measure  of  CILS— TAT’s  impact  on 
allowance  effectiveness  can  be  quantified  simply  by  focusing  analysis  on  changes  in 
source  codes  A  and  C. 
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Table  10.  Analysis  of  Source  Codes  for  Allowance  Effectiveness  Calculation 


T-Test  [P  Value]:  Allowance  Effectiveness  Source  Codes 

-■jP""”  Sourt*  Cade  A 

Bthcnwiu 

H«fl  Grater  Leu  Greater  Leu 

Number 

Analysis  (CILS-TAT  Period) 

Source  Cede  C  Source  Code  D 

Greater  Lets  Greater  Leu 

Source  Code  F 

Greater  Leu 

Source  Code  G 

Greater  Leu 

Source  Code  J 

Greater  Leu 

CG  61 

0.9999 

0.0001 

0.9998 

0.0002 

0.9984 

0.0016 

0.9306 

0.0694 

0.9996 

0.0004 

0.5157 

0.4843 

0.2295 

0.7705 

DDG  53 

0.9988 

0.0012 

0.9994 

0.0006 

0.9997 

0.0003 

0.9951 

0.0049 

0.6573 

0.3427 

0.0036 

0.9964 

0.8045 

0.1955 

fit  cTr,, 01)056 

0.9501 

0.0499 

0.9883 

0.0117 

1.0000 

0.0000 

1.0000 

0.0000 

0.6505 

0.3495 

0.0546 

0.9454 

0.9752 

0.0248 

PValJ  DDG*> 

0.9951 

0.0049 

0.9999 

0.0001 

0.9697 

0.0303 

0.7836 

0.2164 

0.4236 

0.5764 

0.9547 

0.0453 

0.6293 

0.3707 

<0.05  DD0  ‘s 

0.9933 

0.0067 

0.9927 

0.0073 

1.0000 

0.0000 

0.0031 

0.9969 

0.0773 

"A' 

0.0015 

0.9985 

0.2009 

0.7991 

DDG  69 

1.0000 

0.0000 

0.8328 

0.1672 

1.0000 

0.0000 

0.9991 

ii.'Hinv 

04)348 

0.9652 

0.0002 

0.9998 

0.0012 

0.9988 

P  Value  <  0.05  (Average) 

0.9895 

0.0105 

0.9688 

0.0312 

0.9946 

0.0054 

0.7852 

0.2148 

0.4738 

0.5262 

0J550 

0.7450 

0.4734 

0.5266 

CG  67 

0.8848 

0.1152 

0.4458 

0.5542 

1.0000 

0.0000 

0.0059 

0.9941 

0.9928 

0.0072 

0.0133 

0.9867 

0.1142 

0.8858 

CG  70 

0.0870 

0.9130 

0.6767 

0.3233 

0.6379 

0.3621 

0.9818 

0.0182 

0.9958 

0.0042 

0.9230 

0.0770 

0.9925 

0.0075 

CG  72 

0.7416 

0.2584 

0.9361 

0.0639 

0.0000 

0.8464 

0.1536 

0.0841 

0.9159 

0.9038 

0.0962 

0.3629 

0.6371 

CG  73 

0.9391 

0.0609 

0.9982 

0.0018 

0.9886 

0.0114 

0.7845 

0.2155 

0.7052 

0.2948 

0.7088 

0.2912 

0.7205 

0.2795 

Allowance  DDG  55 

0.3263 

0.6737 

0.9453 

0.0547 

0.0146 

0.9854 

0.9935 

0.0065 

0.9979 

0.0021 

1 1  Vv*. 

0.2162 

0.9932 

0.0068 

Effecdvcaets  DDG  60 

0.8793 

0.1207 

0.8238 

0.1762 

1.0000 

0.0000 

0.9614 

0.0386 

0.9986 

0.0014 

0.0297 

0-9703 

0.8825 

0.1175 

P  Value  DDG  61 

0.7788 

0.2212 

0.0087 

0.9913 

0.4629 

0.5371 

0.9959 

0.0041 

0.0945 

0.9055 

0.5556 

0.4444 

0.0114 

0.9886 

>  0.05  DDG  62 

0.8042 

0.1958 

0.7777 

0.2223 

0.3254 

0.6746 

0.3124 

0.6876 

0.9674 

0.0326 

0.0007 

0.9993 

0.8388 

0.1612 

DDG  68 

0.1715 

0.8285 

0.4130 

0.5870 

0.6932 

0.3068 

0.2228 

0.7772 

0.0001 

0.9999 

0.1321 

0.8679 

0.7680 

0.2320 

DDG  70 

0.5246 

0.4754 

0.9993 

0.0007 

0.4345 

0.5655 

0.9936 

0.0064 

0.9851 

0.0149 

1.0000 

0.0000 

0.7420 

0.2580 

DDG  73 

0.6806 

0J 194 

0.2597 

0.7403 

0.6250 

0.3750 

0.0443 

0.9557 

0.9322 

0.0678 

0.5000 

0.5000 

0.0714 

0.9286 

DDG  76 

0.8746 

0.1254 

0.4467 

0.5533 

0.1361 

0.8639 

0.2098 

0.7902 

0.0003 

0.9997 

0.9999 

0.0001 

0.0012 

0.9988 

P  Value  >  0.05  (Average) 

0.6410 

0J590 

0.6442 

0J558 

0.5265 

0.4735 

0.6127 

0.3873 

04*462 

0J538 

0.5459 

0.4541 

0.S415 

0.4585 

%  of  Ships  Statistically 

Significant  Ships  with 

0% 

100% 

0% 

83% 

0% 

100% 

17% 

50% 

17% 

17% 

50% 

17% 

17% 

17% 

P  Val  <  0.05 
%  of  Ships  Statistically 

Significant  Ships  with 

0% 

0% 

8% 

17% 

17% 

25% 

17% 

42% 

17% 

50% 

25% 

17% 

17% 

17% 

P  Val  <  0.05 

%  of  Ships  All  Ships  with 

P  Val  <0.05 

0% 
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Figure  13.  Statistical  Significance  of  Change  in  Allowance  Effectiveness 

(Before  to  After  CILS— TAT) 


The  graphs  in  Figure  13  depict  the  relationship  between  the  observed  changes  in 
data  set  means  and  their  statistical  significance  relative  to  the  p-value  and  the  95%  upper 
confidence  level  (UCL).  The  UCL  would  be  defined  in  this  case  as  a  type  of  interval 
estimate  of  a  population  parameter  and  is  used  to  indicate  the  reliability  of  an  estimate. 
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These  two  graphs  show  that  of  the  data  sets,  which  did  not  show  statistical 
significance,  there  are  some  ships  (Cluster  2)  relatively  close  to  achieving  statistical 
significance.  Relating  Figure  13  back  to  Table  10,  if  the  confidence  interval  was 
increased  to  99%,  the  number  of  statistically  significant  ships  decreases  to  five  (one  CG 
and  four  DDGs).  However,  in  order  to  increase  the  number  of  statistically  significant 
ships  to  90%  of  all  ships  (~16  ships),  the  confidence  interval  must  be  changed  to  30%, 
which  is  well  outside  of  acceptable  limits.  Again,  statistically  significant  means  that  at  a 
95%  confidence  interval  (Type  I  error  <  0.05)  there  is  enough  evidence,  based  on  the 
change  in  means  and  the  variance  in  the  two  data  sets,  to  say  that  there  was  a  change. 
Note  that  there  are  three  ships — CG  70,  DDG  55,  and  DDG  68 — that  experienced  a 
positive  change  in  means,  which  is  opposite  to  the  expected  output.  Herein,  this  shows 
that  while  there  are  some  ships  on  the  cusp  of  achieving  statistical  significance,  only 
marginal  losses  or  gains  are  made  by  increasing  or  decreasing  the  confidence  interval — 
the  ships  that  achieved  statistical  significance  were  well  inside  the  95%  confidence 
interval. 

In  conclusion,  during  the  periods  when  CILS— TAT  was  implemented,  there  were 
six  ships  that  experienced  a  significant  enough  change  in  allowance  effectiveness  to 
result  in  a  Type  I  error  less  than  0.05.  While  the  amounts  obligated  for  each  ship  varied 
significantly,  there  appears  to  be  no  relationship  between  the  amount  obligated  and  the 
resulting  change  in  allowance  effectiveness.  This  result  is  consistent  for  both  ships  that 
experienced  a  statistically  significant  change  and  those  that  did  not.  In  terms  of  the 
parameters  used  to  calculate  allowance  effectiveness,  CILS— TAT  predominately 
impacted  source  codes  A  and  C  with  some  change  to  codes  G  and  J;  however,  not  in  a 
statistically  significant  manner. 

4.  Brownout  Period  Analysis 

Data  analyzed  during  the  brownout  period  covers  December  2009  to  January 
2011.  In  order  to  maintain  unifonnity  of  CILS— TAT  effectiveness  for  those  ships  whose 
24-month  window  continued  past  December  2009,  their  brownout  analysis  window  is 
adjusted.  For  example,  USS  John  S  McCain ’s  brownout  period  started  as  of  June  2009, 
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since  the  24  months  following  CILS— TAT  implementation  did  not  complete  until  June 
2009.  This  ensures  that  all  ships’  data  are  given  the  full  24-month  window  following 
CILS— TAT  implementation. 

During  the  brownout  period,  there  was  a  considerable  decrease  in  allowance 
effectiveness  across  the  board.  Looking  at  the  two  data  sets  within  the  before  data,  the 
reported  allowance  effectiveness  ranges  from  a  minimum  of  25.30%  to  a  maximum  of 
98.40%  with  an  overall  average  of  71.12%  and  a  standard  deviation  of  13.28%.  In  terms 
of  ships  grouped  by  class,  there  is  not  an  appreciable  difference  in  the  before  data  sets. 
Among  the  after  data  sets,  allowance  effectiveness  ranges  from  a  minimum  of  0.00%  to  a 
maximum  of  100.00%  with  an  overall  average  of  66.35%  and  a  standard  deviation  of 
13.96.  Table  1 1  summarizes  the  changes  in  the  means  data  sets  from  before  to  after.  To 
be  consistent  for  comparison  purposes  with  the  change  in  means  calculated  in  the  Welch 
Two-Sided  t-test,  the  change  in  means  calculation  is  based  on  the  before  data  set  mean 
minus  the  after  data  set  mean. 


Table  1 1 .  Summary  of  Allowance  Effectiveness  (Before  to  After  Brownout  Period) 


Summary  Statistic*:  Allowance  Effectiveness  [Brownout  Period! 

Before 

Max  Min  Average  StdDcv 

Number 

Variance 

Max 

Min 

After 

Average 

StdDcv 

Variance 

Change  in 
Mean 

CG 

61 

97.60 

59.00 

8033 

9.15 

83.78 

90.60 

40.00 

r."  3 . 

12.66 

160.17 

12.93 

CG 

67 

91.50 

45.90 

73.79 

10.49 

110.07 

91.00 

41.10 

67.94 

12.73 

161.95 

5.85 

CG  CG 

70 

79.50 

41.10 

65.55 

11.03 

121.75 

89.10 

39.80 

57.64 

1231 

148.98 

7.90 

CG 

72 

91.70 

25.30 

75.88 

13.79 

190.13 

93.00 

48.70 

73.57 

10.46 

109.49 

2.31 

CG 

73 

93.60 

48.60 

74.42 

10.96 

120.22 

89.80 

4630 

66.52 

10.36 

107.25 

7.90 

CG  Overall  [Avg| 

97.60 

25.30 

73.97 

12.00 

143.99 

93.00 

39.80 

67.25 

12.49 

156.05 

6.72 

DDG 

53 

90.70 

44.70 

71.01 

15.11 

22839 

100.00 

0.00 

62.41 

21.39 

457.74 

8.60 

DDG 

55 

93.40 

46.60 

66.32 

12.57 

157.94 

87.90 

19.20 

63.72 

1730 

295.82 

2.60  1 

DDG 

56 

89.60 

41.10 

71.64 

12.48 

155.71 

80.90 

31.40 

65.45 

13.14 

172.64 

6.18 

.......  DDG 

All  Ships 

59 

85.10 

53.30 

71.87 

10.31 

10634 

90.60 

35.20 

73.09 

11.44 

130.77  1 

-132  | 

r  DDG 

60 

89.00 

28.70 

67.10 

15.66 

145  36 

83.90 

25.90 

53.81 

11.43 

130.70 

13.29 

DDG 

61 

95.70 

29.00 

67.52 

14.50 

210.18 

92.60 

26.70 

63.15 

12.43 

154.60 

4.37 

DDG  DDG 

62 

92.50 

46.00 

67.13 

12.95 

167.82 

92.70 

34.50 

66.72 

13.86 

192.05 

0.42 

DDG 

65 

92.40 

38.40 

73.69 

11.45 

131.01 

100.00 

25.00 

60.61 

14.79 

218.82 

13.08 

DDG 

68 

82.70 

48.40 

64.30 

10.00 

100.01 

90.90 

12.00 

65.31 

14.02 

196.45 

-1.01  | 

DDG 

69 

98.40 

69.50 

8436 

7.96 

63.38 

94.90 

55.60 

73.34 

11.09 

123.08 

10.92 

DDG 

70 

91.20 

43.40 

72.83 

13.17 

173.53 

88.70 

29.60 

69.28 

12.87 

165.69 

3.55  | 

DDG 

73 

92.30 

45.10 

69.10 

11.99 

143.76 

95.30 

36.00 

71.49 

1134 

126.39 

-2J9 

DDG 

76 

92.00 

29.00 

63.53 

16.04 

257.41 

88.10 

32.50 

65.85 

12.15 

147.52 

-2.33 

DDG  Overall  IAvrI 

98.40 

28.70 

70.02 

13.60 

184.83 

100.00 

0.00 

66.00 

14.49 

209.87 

4.03 

All  Ships  Overall 

98.40 

2530 

71.12 

1338 

176.26 

100.00 

0.00 

66J5 

13.96 

194.74 

4.77 
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In  terms  of  statistical  significance  for  the  brownout  period,  the  expected  result 
was  that  the  mean  of  the  before  data  set  would  be  greater  than  the  mean  of  the  after  data 
set.  Based  on  this  expected  movement,  the  greater  hypothesis  was  used  as  the  base  for 
analyzing  the  brownout  period.  Again  using  the  Welch  Two-Sided  /-test,  10  ships  (four 
CGs  and  six  DDGs)  saw  a  statistically  significant  change  in  the  means  of  the  before  and 
after  data  sets.  On  average,  all  of  the  ships  experienced  a  change  of  4.77.  When  grouped 
together,  CGs  and  DDGs  experienced  an  average  change  of  6.72  and  4.03,  respectively, 
in  the  data  sets. 

In  stark  contrast  to  the  change  experienced  during  the  CILS— TAT  period,  during 
the  brownout  period,  at  a  confidence  interval  of  99%  (Type  1  error  <  0.01),  there  is 
enough  evidence  to  prove  that  six  ships  experienced  the  expected  change  in  allowance 
effectiveness.  There  are,  however,  four  ships  that  experienced  a  small,  negative  change 
in  mean,  resulting  in  very  low  statistical  significance.  For  the  brownout  period,  in  order 
to  capture  90%  of  the  ships  (~16  ships)  being  statistically  significant,  the  confidence 
interval  would  need  to  shift  to  60%.  Table  12  summarizes  the  /-test  results. 


Table  12.  /-Test  Summary  of  Allowance  Effectiveness 


T-Test:  Allowance  Effectiveness  (Brownout] 

Mean  of  Mean  of 

Before  After 

Greater  Alternative  -  // 

Change  In _ ..  .. 

Mean  TSUtl!tlc 

P  Value 

t  ", 

DOF 

LCL 

UCL 

CG  61 

80.23 

66.14 

14.10 

5.02 

0.000003 

57.48 

9.41 

-*-00 

CG  67 

73.79 

67.99 

5.80 

1.97 

0.027090 

54.12 

0.87 

-*-00 

CG  70 

65.55 

57.64 

7.90 

2.57 

0.006508 

52.56 

2.76 

-*-oo 

CG  72 

75.88 

72.16 

3.72 

1.15 

0.128849 

37.77 

-1.74 

t-oo 

CG  73 

74.42 

63.49 

10.93 

4.29 

0.000063 

36.60 

6.63 

-t-oo 

DDG  53 

71.01 

61.63 

9.38 

1.94 

0.028761 

57.98 

1.29 

+  00 

DDG55 

66.32 

63.72 

2.60 

0.68 

0.251142 

57.44 

-3.83 

+  00 

DDG  56 

71.64 

65.45 

6.18 

1.78 

0.040484 

50.76 

0.37 

■♦•oo 

DDG  59 

Individual _ „ 

71.87 

73.48 

-1.61 

-0.59 

0.722637 

49.29 

-6.16 

+  00 

DDG  60 

67.10 

54.01 

13.09 

3.51 

0.000575 

39.31 

6.80 

-*-  00 

DDG  61 

67.52 

61.15 

6.37 

1.77 

0.042202 

44.04 

0.31 

-*-oo 

DDG  62 

67.13 

67.18 

-0.05 

-0.01 

0.505766 

50.14 

-5.82 

-*-  00 

DDG  65 

73.69 

60.49 

13.20 

3.89 

0.000134 

56.59 

7.52 

-t-  00 

DDG  68 

64.30 

62.95 

1.35 

0.43 

0.335418 

57.90 

-3.94 

-TOO 

DDG  69 

84.26 

73.34 

10.92 

4.33 

0.000032 

55.00 

6.69 

+  00 

DDG  70 

72.83 

68.17 

4.66 

1.41 

0.083385 

45.05 

-0.91 

-*-oo 

DDG  73 

69.10 

70.28 

-1.18 

-0.38 

0.647678 

47.61 

-6.37 

-*-oo 

DDG  76 

63.53 

64.15 

-0.63 

-0.16 

0.564648 

39.57 

-7.08 

+  00 
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Cluster  analysis  was  again  used  to  discover  the  reason  that  these  10  ships 
experienced  a  statistically  significant  change.  Figure  14  shows  that  during  the  brownout 
period,  ships  in  Cluster  1,  similar  to  the  CILS— TAT  period  results,  had  the  largest 
changes  in  source  codes  A,  C,  G,  and  J.  However,  during  the  brownout  period,  source 
codes  G  and  J — vice  A  and  C — had  a  Type  I  error  less  than  0.05.  Looking  at  the 
magnitude  of  the  change  in  means  among  the  source  codes,  it  is  clear  that  source  code  J 
dominated  the  movement  of  allowance  effectiveness.  For  comparison,  source  code  J’s 
ellipse  was  centered  on  a  change  in  means  of  -1.31  during  the  CILS— TAT  period  and  on  - 
39.96  during  the  brownout  period;  source  code  A’s  ellipse  was  centered  on  -57.73  during 
the  CILS— TAT  period  and  on  18.90  in  the  brownout  period.  During  the  brownout  period, 
both  the  magnitude  and  statistical  significance  flipped.  Looking  at  the  parallel  coordinate 
plot  at  the  top  of  the  figure,  note  that  during  the  brownout  period,  the  higher  the  value, 
the  better  for  the  change  in  means  of  source  codes  A,  C,  D,  and  F  and  the  smaller  the 
number,  the  better  for  the  change  in  means  of  source  codes  G  and  J.  Based  on  this 
analysis,  it  is  clear  that  the  change  in  source  code  J  is  the  reason  that  the  10  ships 
experienced  a  statistically  significant  change  in  allowance  effectiveness. 
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Source  Code  Description 


Standardized  Allowance  Effectiveness  and  Variable  Clusters  Means 


A 

Allowance  List  Material  issues 
from  storeroom  stock 

C 

Non-Allowance  List  Material 
(SIM)  issues  from  storeroom  stock 

D 

Allowance  List  Material  Not  In 

Stock  (NIS)  when  requested 

F 

Non-Allowance  List  Material 
(SIM)  Not  In  Stock  (NIS)  when 
requested 

G 

Not  Carried  (NC)  repair  parts 
which  arc  not  listed  on  an  APL  in 
the  ship's  COS  file 

J 

Not  Carried  (NC)  repair  parts 
which  arc  listed  on  an  APL.  but 
does  not  compute  for  an  allowance 

Cluster  Location  &  95%  Confidence  Ellipse 
Cluster  Number: 


P  Value:  Allowance  Effectiveness 


Change  in  Means:  Allowance  Effectiveness • 


Source  Code  A 
Source  Code  C 
Change  in  Source  Code  D 
Means  Source  Code  F 
Source  Code  G 
Source  Code  J 


Cluster  Standard  Deviation 


—  ■  • 

-I  -  • 

0.009361 

0.139951 

0.378823 

0.014639 

0.111190 

0.251522 

10.26 

5.99 

1.58 

3.05 

3.39 

2.79 

18.09 

34.64 

-16.49 

31.28 

4.13 

29.62 

5.99 

3.09 

-5.52 

3.63 

5.47 

4.10 

-15.83 

4.55 
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Figure  14.  Impact  of  Source  Codes  on  Allowance  Effectiveness  Using  Cluster  Analysis 


In  conclusion,  it  is  clear  that  the  brownout  period  caused  an  impact  on  a  larger 
number  of  ships  when  compared  to  the  CILS— TAT  period.  However,  looking  at  the 
bigger  picture  and  Figure  15  which  shows  allowance  effectiveness  and  the  standard 
deviation  for  the  three  different  periods,  it  is  clear  that  the  brownout  period  resulted  in 
many  of  the  ships  returning  to  their  pre-CILS— TAT  means  and  standard  deviations. 
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Variance  in  Allowance  Effectiveness  Before  to  After  Brownout 
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Figure  15.  During  the  Brownout  Period,  Many  Ships  Return 
to  Before  CILS-TAT  Levels 


In  summary,  five  ships  were  statistically  significant  during  both  the  CILS-TAT 
period  and  the  brownout  period.  Of  those  five  ships,  three  ended  up  with  allowance 
effectiveness  means  that  were  higher  than  their  pre-CILS— TAT  levels,  indicating  a 
lasting  effect  of  the  implementation.  Six  ships  were  not  significant  during  the  CILS- 
TAT  analysis  but  returned  to  within  10%  of  their  pre-CILS— TAT  mean  during  the 
brownout  phase.  This  change  is  not  directly  related  to  CILS-TAT,  although  it  is  a 
variable  to  consider.  The  remaining  seven  ships  did  not  see  a  significant  change  in  mean 
in  either  of  the  analysis  windows. 
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V.  CONCLUSIONS  AND  RECOMMENDATIONS 


A.  CONCLUSIONS 

In  summary,  the  goal  of  this  research  was  to  conduct  a  BCA  of  CILS— TAT  to 
determine  its  historical  effectiveness  against  allowancing  metrics.  The  analysis  examined 
the  before  and  after  periods  to  detennine  if  the  benefits  were  substantial  enough  to  restart 
the  program  following  the  implementation  of  the  Navy’s  ERP.  The  research  identified 
three  specific  questions  used  in  guiding  the  process.  In  the  following  paragraphs,  we 
summarize  our  findings. 

First,  we  asked  the  question  of  how  did  CILS— TAT  impact  allowance 
effectiveness  for  our  sample  and  at  what  cost?  The  findings  suggested  that  CILS— TAT 
had  a  direct  and  positive  impact  on  allowance  effectiveness  for  six  out  of  18  ships.  Of 
the  12  remaining  ships,  nine  saw  a  positive  change  in  allowance  effectiveness,  although 
this  change  is  not  statistically  significant  and  thus  cannot  be  attributed  to  CILS— TAT. 
Based  on  these  findings,  CILS— TAT  does  achieve  the  ends  for  which  it  is  designed. 
Regardless  of  whether  the  U.S.  Navy  re-implements  this  program  or  some  other  targeted 
allowancing  technique,  we  have  found  that  there  is  a  need  throughout  the  fleet  for  this 
model. 

Second,  we  explored  the  question  of  whether  there  are  correlations  between 
allowance  effectiveness  rates,  POTF  from  CASREPs,  and  CILS— TAT  cost.  Our  research 
concluded  that  there  was,  in  fact,  no  direct  correlation  between  the  effectiveness  of 
CILS— TAT  and  these  three  variables.  Of  the  ships  that  were  statistically  significant,  their 
performance  as  related  to  allowance  effectiveness  and  POTF  could  not  be  linked  directly 
to  the  amount  of  funds  obligated. 

Last,  we  sought  to  answer  the  question  of  the  effect  of  not  having  CILS— TAT 
during  2009-2011,  when  traditional  allowancing  procedures  were  not  available.  The 
research  demonstrated  that  in  the  absence  of  targeted  allowancing  techniques,  ships  that 

received  CILS— TAT  were  able  to  maintain  allowance  effectiveness  rates  within  10%  of 
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their  pre-CILS— TAT  averages  for  up  to  24  months  following  implementation.  These 
results  are  most  encouraging  since  they  display  the  longer  term  impact  of  CILS— TAT  as 
well  as  provide  an  expected  shelf  life  for  each  iteration  of  the  process. 

Upon  concluding  our  research,  we  were  able  to  determine  that  CILS— TAT  did 
directly  contribute  to  the  improved  allowance  effectiveness  of  ships  within  the  BMD 
fleet.  Ships  in  the  sample  that  saw  increases  in  allowance  effectiveness  were  able  to 
maintain  those  higher  levels  of  performance  for  up  to  two  years  in  both  the  presence  and 
absence  of  traditional  allowance  maintenance  processes.  While  CILS— TAT  did  not 
improve  allowance  effectiveness  to  the  same  extent  on  every  ship,  the  fact  that  it  did 
improve  one  third  of  our  sample  cannot  be  ignored,  and  we  recommend  that  the  Navy 
continue  to  utilize  this  program  to  improve  allowance  effectiveness  in  the  future. 

1.  Recommendations  for  Change 

How  can  the  effectiveness  of  a  CILS— TAT  product  be  measured?  This  depends 
on  the  priority  of  the  decision-maker  as  to  whether  CILS— TAT  should  be  used  to 
primarily  improve  allowance  effectiveness  with  a  second  order  effect  of  reducing  the 
number  of  CASREPs  or  whether  the  program  should  first  reduce  CASREP  frequency  and 
then  focus  on  allowance  effectiveness  if  resources  exist.  We  believe  that  the  system 
could  be  tailored  in  either  direction. 

CILS— TAT  seeks  to  improve  allowance  effectiveness  first  with  no  consideration 
given  to  the  priority  of  critical  systems.  Allowances  added  as  a  result  of  CILS— TAT  are 
funded  on  an  all-or-nothing  basis,  and  when  funding  is  constrained,  use  of  a  knapsack 
model  would  aid  in  optimizing  dollars  spent.  For  example,  funding  could  be  applied  to 
all  9X  Cognizance  Code  (COG)  items  with  MCCs  2,  3,  or  4  first  since  this  family  of 
COGs  accounts  for  34.76%  of  all  CASREPs  recorded  and  only  3.02%  of  total  CASREP 
cost.  Additionally,  MCCs  2,  3,  and  4  accounted  for  75.17%  of  all  CASREPs  during  the 
analysis  window.  If  more  priority  were  given  to  these  MCCs,  the  same  effect  on  overall 
system  readiness  could  be  realized  at  half  of  the  cost.  Over  the  time  period  analyzed, 

CILS— TAT  requisitions  were  funded  at  the  rates  shown  in  Table  13: 
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Table  13.  Breakdown  of  CILS— TAT  Requisition  Cost  (2003-2009) 


CILS-TAT  Requisition  Cost 

1 

Cognizance  Code  (COG) 

3  7  9 

Grand  Total 

1 

0.53% 

0.65% 

33.95% 

4.15% 

39.27% 

2 

0.06% 

0.31% 

10.52% 

1.94% 

12.84% 

MCC 

0.10% 

0.21% 

25.05% 

1.88% 

27.25% 

4 

0.03% 

0.06% 

2.63% 

0.39% 

3.10% 

5 

0.00000% 

0.00000% 

0.00000% 

0.00005% 

0.00005% 

N/A 

0.74% 

0.49% 

12.13% 

4.19% 

17.54% 

Grand  Total 

1.44% 

1.72% 

84.29% 

12.55% 

100.00% 

As  shown  in  Table  13,  nearly  40%  of  all  funding  applied  as  a  result  of  CILS— 
TAT  is  used  for  NIINs  with  MCC  1  (failure  results  in  minor  mission  impact).  By  using  a 
nearly  identical  amount  of  funding  across  MCCs  2,  3,  and  4,  a  much  larger  impact  could 
be  realized  on  the  ship’s  ability  to  perform  its  mission. 
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Figure  16.  Breakdown  of  CASREP  Frequency 
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Figures  16  and  17  clearly  demonstrate  the  effect  of  making  a  change  in  funding 
priorities  by  using  MCCs  as  a  factor.  Results  are  based  on  the  analysis  of  13,956 
CASREPs  submitted  by  sample  ships  during  the  analysis  window. 

2.  Potential  Obstacles 

The  primary  obstacle  to  changing  the  implementation  of  CILS— TAT  would  be  in 
making  the  shift  away  from  allowance  effectiveness  as  a  primary  measure,  to  an  approach 
focused  more  on  critical  systems.  In  a  perfect  world,  all  systems  would  operate  as 
required  at  all  times,  but  in  reality,  certain  systems  will  experience  difficulty  and  operate 
in  a  degraded  capacity.  For  this  reason,  it  is  important  that  we  focus  more  closely  on 
critical  systems  both  during  the  initial  allowance  provisioning  process  and  during  times 
when  allowances  are  being  maintained.  By  shifting  to  an  MCC-based  allocation  model, 
we  would  be  moving  away  from  some  of  the  more  traditional  supply  effectiveness 
metrics.  Since  this  change  would  be  both  cultural  and  procedural  in  nature,  it  makes 
implementation  the  most  difficult. 
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This  change  would  result  in  a  decrease  in  allowance  effectiveness  but  an  increase 
the  ability  of  a  ship  to  carry  out  mission  tasking.  In  an  era  where  the  DoD  is  expected  to 
operate  at  prior  years’  levels  with  fewer  resources,  perhaps  this  cultural  change  is 
necessary. 

B.  RECOMMENDATIONS  FOR  FUTURE  RESEARCH 

During  the  course  of  our  research,  we  identified  some  areas  where  our  research 
could  be  improved  or  additional  questions  could  be  answered  through  the  use  of  a  larger 
sample  size  as  well  as  a  more  diverse  set  of  platforms  selected.  An  additional 
improvement  to  this  research  would  be  to  select  a  control  group,  or  in  this  case,  a  control 
ship.  This  ship  would  be  evaluated  over  the  same  time  period;  however,  it  would  not 
receive  a  CILS— TAT.  Its  performance  could  then  be  compared  against  several  other 
ships  of  the  same  class  or  mission  area  to  detennine  the  long-term  benefits  of  the 
program  at  the  unit  level. 

There  are  other  variables  that  could  also  contribute  to  the  supply  effectiveness  of 
a  ship  that  were  not  evaluated  here.  Particularly,  if  supply  gross  and  net  effectiveness  are 
to  be  evaluated,  then  operational  funding  has  to  be  considered  as  well.  Improved 
operational  funding  levels  will  increase  the  opportunity  for  a  ship  to  replenish  its  stock 
and  thereby  reduce  the  number  of  Not  in  Stock  (NIS)  demands.  Our  research  evaluated 
the  NC  rate  for  each  ship  and  was  therefore  independent  of  periodic  OPTAR  funding 
grants. 

A  study  should  be  conducted  comparing  the  allowance  maintenance  done  through 
traditional  processes  as  well  as  through  CILS— TAT.  We  were  unable  to  gain  access  to 
this  data,  but  we  hypothesize  that  those  ships  in  our  study  that  did  not  see  a  significant 
statistical  change  attributed  directly  to  CILS— TAT  may  have  been  a  result  of  increased 
traditional  allowance  products  (e.g.,  more  allowance  adds  through  ASIs).  The  results  of 
this  new  study  would  provide  a  more  accurate  understanding  as  to  the  precise  impact  of 
CILS-TAT. 
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Lastly,  in  order  to  grasp  CILS— TAT’s  impact  on  CASREP  frequency,  additional 
analysis  could  be  conducted  on  NIINs  added  as  a  result  of  CILS— TAT.  Since  the 
allowances  added  to  a  ship  following  a  CILS— TAT  are  derived  from  previously  NC 
demands,  a  percentage  of  those  demands  were  recorded  as  CASREPs.  Analysis  could  be 
conducted  to  determine  the  demand  frequency  of  previous  CASREP  NIINs  that  are  added 
to  the  ship’s  stock  and  subsequently  avoid  a  new  CASREP. 
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